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ABSTRACT 


This  study  examined  the  Naval  Facilities  Engineering 
Command's  current  use  of  construction  contractor  performance 
evaluations  (SF  1420)  from  the  viewpoint  of  accepted  control 
and  measurement  theory.  Surveys  of  field  contract  adminis¬ 
trators,  supervisory  civil  engineers,  field  contract  spe¬ 
cialists,  and  construction  contractors  were  completed  to 
assess  their  use  of  and  views  about  the  evaluations. 

The  study  concluded  that:  (1)  NAVFAC  lacks  standards  of 
performance  to  describe  the  distinction  between  satisfac¬ 
tory,  outstanding,  and  unsatisfactory  performances  (2)  Con¬ 
tractors  are  not  generally  aware  of  the  evaluation  process; 
(3)  Evaluations  are  not  used  to  provide  contractors  feed¬ 
back;  (4)  Evaluators  are  not  well  trained;  (5)  Evaluations 
are  not  fully  utilized;  (6)  The  data  base  of  evaluation 
information  is  inadequate. 

The  study  recommends  that;  (1)  NAVFAC  issue  a  policy 
statement  to  contractors  to  clarify  the  evaluations  uses, 
standards,  and  performance  elements;  (2)  interim  evaluations 
be  issued  to  provide  contractors  feedback;  (3)  contractors 
receive  copies  of  all  their  evaluations;  (4)  evaluators 
receive  uniform  training  on  completion  of  evaluations;  (5) 
SF  1420  be  modified  to  allow  for  a  more  specific  evaluation. 
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I .  INTRODUCTION 

The  purpose  of  this  study  is  to  examine  the  Naval 
Facilities  Engineering  Command's  (NAVFAC)  current  use  of 
construction  contractor  performance  evaluations  (Standard 
Form  1420)  .  That  current  use  will  then  be  analyzed  using 
accepted  theory  about  control  systems,  measurements,  and 
performance  appraisals.  Based  on  the  analysis  some  recom¬ 
mended  improvements  to  NAVFAC ' s  contractor  performance 
evaluation  system  will  be  made.  Appendix  A  defines  some  of 
the  key  words  used  throughout  this  study. 

A.  RESEARCH  QUESTIONS 

The  primary  research  question  is:  how  can  NAVFAC  make 
better  use  of  the  construction  contractor  performance  evalu¬ 
ations?  These  secondary  research  questions  will  also  be 
addressed : 

1.  How  does  NAVFAC  currently  use  the  performance 
evaluations? 

2.  What  does  the  Federal  Acquisition  Regulation  (FAR)  say 
about  the  contractor  performance  evaluations? 

3.  What  amount  of  attention  do  NAVFAC ' s  managers  place  on 
the  evaluations? 

4.  Do  the  contractors  have  a  formal  system  for  evaluating 
their  subcontractors? 

5.  What  do  contractors  think  about  the  performance  evalu¬ 
ation  process? 

6.  What  alternative  uses  are  there  for  the  evaluations? 


7.  What  percentage  of  contracts  result  in  outstanding, 
satisfactory,  and  unsatisfactory  evaluations. 

B.  METHODOLOGY  AND  LITERATURE  REVIEW 

Mail  and  telephone  surveys  were  used  to  determine  the 
current  uses  of  and  attitudes  about  contractor  performance 
evaluations.  Different  mail  surveys  were  sent  to: 

1.  NAVFAC ' s  Assistant  Resident  Of f icers-in-Charge  of  Con¬ 
struction  (AROICC's)  who  directly  administer  construc¬ 
tion  contracts. 

2 .  Construction  contractors  who  have  previously  done  work 
for  NAVFAC. 

3.  Supervisory  civil  engineers  who  manage  NAVFAC ' s  field 
offices . 

Field  contract  specialists  were  surveyed  by  telephone. 

Literature  on  control  and  measurement  systems  was 
reviewed  and  provided  a  body  of  theory  which  was  used  to 
analyze  the  research  data.  The  FAR,  Department  of  Defense 
FAR  Supplemental  (DFARS) ,  and  NAVFAC's  Contracting  Manual 
(P-68)  were  used  to  develop  background  on  the  requirement 
for  issuing  contractor  performance  evaluations. 

C.  LIMITATIONS 

A  search  for  literature  about  other  contractor  perform¬ 
ance  evaluation  systems  produced  no  relevant  information  for 
this  study.  Some  data  located  at  NAVFAC's  Engineering  Field 
Divisions  (EFD)  could  not  be  provided  due  to  the  heavy 
seasonal  workload  demands.  The  survey  population  sizes  were 
relatively  small.  The  limitations  on  the  use  of  data 


9 


generated  from  these  small  populations  are  discussed  in 
Chapter  IV. 


D.  SUMMARY  OF  FINDINGS  AND  RECOMMENDATIONS 

This  writer  concluded  that  NAVFAC  could  significantly 
improve  its  use  of  the  contractor  performance  evaluations. 
These  recommendations  for  improvements  are  made: 

1.  Issue  a  policy  statement  describing  to  contractors  the 
purpose  for  the  evaluation  system. 

2.  Issue  contractors  interim  evaluations  mid-way  through 
the  contract. 

3.  Provide  contractors  copies  of  their  final  evaluations. 

4.  Provide  AROICC's  training  on  how  to  complete  the 
evaluations . 

5.  Increase  management  emphasis  on  the  evaluation 
process . 

6.  Provide  meaningful  rewards  to  contractors  who  are 
rated  outstanding. 

7.  Publicize  the  times  evaluations  are  successfully  used 
to  deny  contract  awards  to  contractors  with  unsatis¬ 
factory  performance  records. 

8.  Improve  the  access  to  and  use  of  contractor  perform¬ 
ance  evaluation  information. 

9.  Improve  the  Standard  Form  1420. 


E.  ORGANIZATION 

The  rest  of  this  study  consists  of: 

-  Chapter  II--Background  information  about  NAVFAC' s 
contracting  environment  and  the  regulatory  requirements 
for  contractor  performance  evaluations. 

-  Chapter  III--Control ,  measurement,  and  performance 
appraisal  theory,  and  three  models  of  possible  uses  for 
the  evaluations. 


Chapter  IV--The  research  data  summary. 

Chapter  V--An  analysis  of  contractor  performance  evalua 
tions  as  a  control  mechanism. 

Chapter  VI--Conclusions  and  recommendations 

Supporting  appendices. 
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II .  BACKGROUND 


This  chapter  provides  background  information  about: 


1. 

NAVFAC 's  construction  mission  and 

environment 

contracting 

2  . 

NAVFAC ' s  organization 

3  . 

The  Federal  Acquisition  Regulation  (FAR)  requirements 
for  pre-award  surveys  and  performance  evaluations 

4  . 

Standard  Form  1420,  Performance 

Construction  Contract 

Evaluation- 

5. 

The  Small  Business  Administration  (SBA) 
tractor  responsibility  determinations 

role  in  con- 

A.  NAVFAC  BACKGROUND 

The  Navy  Facilities  Engineering  Command  maintains  the 
U.S.  Navy  shore  facilities  throughout  the  world.  NAVFAC 
employs  more  than  25,000  people  to  ensure  its  mission  is 
properly  performed.  NAVFAC  augments  its  forces  by  contract¬ 
ing  out  more  than  $2  billion  worth  of  construction  to 
civilian  firms  each  year.  NAVFAC' s  goal  is  to  ensure  each 
contract  is  completed  safely,  on  time,  within  cost  and  at 
satisfactory  quality. 

To  accomplish  that  goal  NAVFAC  is  organized  on  the 
levels  shown  in  the  diagram  below: 
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NAVFAC  Headquarters  provides  overall  guidance,  makes 
policy  decisions,  and  provides  staff  support  to  the 
Engineering  Field  Divisions  ( EFD) .  Each  EFD  awards  con¬ 
struction  contracts,  monitors  contract  progress,  and  assists 
field  offices  in  their  contract  administrative  efforts.  The 
field  offices  make  some  contract  awards  and  administer  the 
construction  contracts.  Each  field  office  is  managed  by  a 
supervisory  civil  engineer.  Contract  specialists  award  the 
contracts  and  provide  expertise  in  contractual  matters.  The 
contract  administration  is  performed  by  civilian  contract 
administrators  and  by  Civil  Engineer  Corps  Officers  (Assis¬ 
tant  Resident  Of f icers-in-Charge  of  Construction,  AROICC's). 
Technical  inspection  of  construction  work  is  performed  by 
construction  specialists. 

Almost  all  NAVFAC  construction  contracts  are  awarded 
using  a  publicly  advertised  sealed  bid  process.  Under  this 


process  NAVFAC  advertises  its  prospective  contracts  using  an 


invitation  for  bids.  Contractors  compete  for  the  contract 


award  by  submitting  sealed  firm-fixed-price  bids.  The 
contract  is  awarded  to  the  lowest  responsive  responsible 
bidder.  To  be  responsive  a  bidder  must  meet  all  the  condi¬ 
tions  of  the  bid  invitation.  To  be  determined  responsible  a 
bidder  must  be  able  to  demonstrate  the  capacity  and  capabil¬ 
ity  to  accomplish  the  contract  work.  NAVFAC  has  little  con¬ 
trol  over  who  its  low  bidders  are.  It  does  have  some 
control  in  determining  whether  a  contractor  is  deemed 
responsible.  Responsibility  determinations  are  made  during 
a  pre-award  survey.  The  next  section  describes  the  regula¬ 
tory  requirements  concerning  pre-award  surveys  and  contrac¬ 
tor  performance  evaluations. 

B.  REGULATORY  REQUIREMENTS 
1 .  Pre-Award  Surveys 

During  a  Pre-Award  survey,  a  contract  specialist 
will  examine  the  apparent  low  bidders  qualifications  to 
determine  whether  or  not  that  contractor  is  responsible.  To 
be  determined  responsible  a  prospective  contractor  must  meet 
several  criteria,  including  "Have  a  satisfactory  performance 
record."  (FAR,  1984,  par.  9.104(c))  Contracts  shall  be 
awarded  to  responsible  prospective  contractors  only."  (FAR, 
1984,  par.  9.103(a))  The  determination  of  a  contractor's 
responsibility  is  important  because: 

The  award  of  a  contract  to  a  supplier  based  on  lowest 
evaluated  price  alone  can  be  false  economy  if  there  is 
subsequent  default,  late  deliveries,  or  other 
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unsatisfactory  performance  resulting  in  additional 
contractual  or  administrative  costs.  While  it  is  impor¬ 
tant  that  Government  purchases  be  made  at  the  lowest 
price,  this  does  not  require  an  award  to  a  supplier  solely 
because  that  supplier  submits  the  lowest  offer.  A  pros¬ 
pective  contractor  must  affirmatively  demonstrate  its 
responsibility,  including  when  necessary,  the 
responsibility  of  its  proposed  subcontractors.  (FAR, 
1984,  par.  9.103(c)) 

2 .  Contractor  Performance  Evaluations 

A  major  source  of  information  about  a  contractor's 

past  performance  is  their  contractor  performance  evaluations 

from  prior  contracts.  Evaluation  of  contractor  performance 

is  a  special  aspect  of  contracting  for  construction.  The 

FAR  says : 

(b)  Preparation  of  Performance  Evaluation  Reports: 

(1)  The  contracting  activity  shall  evaluate  contractor 
performance  and  prepare  a  performance  report  for  each  con¬ 
struction  contract  of 

(i)  $500,000  or  more; 

(ii)  $100,000  or  more,  if  any  element  of  per¬ 
formance  was  either  unsatisfactory  or 
outstanding; 

(iii)  More  than  $10,000,  if  the  contract  was  ter¬ 
minated  for  default;  or 

(iv)  $100,000  or  more,  if  the  contract  was  ter¬ 
minated  for  the  convenience  of  the 
Government. 

(2)  The  report  shall  be  prepared  at  the  time  of  final 
acceptance  of  the  work,  at  the  time  of  contract  termina¬ 
tion,  or  at  other  times,  as  appropriate,  in  accordance 
with  agency  procedures.  Ordinarily,  the  evaluating  offi¬ 
cial  who  prepares  the  report  should  be  the  person  respon¬ 
sible  for  monitoring  contract  performance. 

(3)  If  the  evaluating  official  concludes  that  a  con¬ 
tractor's  overall  performance  was  unsatisfactory,  the  con¬ 
tractor  shall  be  advised  in  writing  that  a  report  of 
unsatisfactory  performance  is  being  prepared  and  the  basis 
for  the  report.  If  the  contractor  submits  any  written 
comments,  the  evaluating  official  shall  include  them  in 
the  report,  resolve  any  alleged  factual  discrepancies,  and 
make  appropriate  changes  in  the  report. 

(4)  The  head  of  the  contracting  activity  shall 
establish  procedures  which  ensure  that  fully  qualified 
personnel  prepare  and  review  performance  reports. 
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(b)  Review  of  performance  reports.  Each  performance 
report  shall  be  reviewed  to  ensure  that  it  is  accurate  and 
fair.  The  reviewing  official  should  have  knowledge  of  the 
contractor's  performance  and  should  normally  be  at  an 
organizational  level  above  that  of  the  evaluating 
official . 

(c)  Distribution  and  use  of  performance  reports.  (1) 
Each  performance  report  shall  be  distributed  in  accordance 
with  agency  procedures.  One  copy  shall  be  included  in  the 
contract  file.  The  contracting  activity  shall  retain  the 
report  for  at  least  six  years  after  the  date  of  the 
report . 

(2)  Before  making  a  determination  of  responsibility  in 
accordance  with  Subpart  9.1,  tl  .  contracting  officer  may 
consider  performance  reports  in  accordance  with  agency 
instructions.  (FAR,  1984,  par.  36.201) 

The  Department  of  Defense  FAR  Supplement  mandates 
that  "Performance  evaluations  of  construction  contractors 
shall  be  used  in  making  responsibility  determinations." 
( DFARS ,  1934,  par.  36.201(c))  NAVFAC ' s  P-68  offers  this 
guidance  about  contractor  performance  evaluations: 

PERFORMANCE  REPORT.  Contractor  performance  reports 
are  valuable  contract  records  and  should  be  prepared  by 
qualified  personnel  in  a  careful  and  conscientious  man¬ 
ner.  These  reports  must  be  based  on  factual  rather  than 
subjective  data.  These  reports  frequently  form  the  basis 
for  the  selection  of  contractors  for  the  accomplishment  of 
critical  work.  They  are  essential  in  findings  of  nonre¬ 
sponsibility  for  contractors  that  have  done  prior  work  for 
NAVFACENGCOM.  OICC's  are  responsible  for  assuring  evalua¬ 
tion  reports  are  promptly  and  accurately  completed  and 
distributed.  Unless  the  contractor  correctly  points  out 
factual  errors,  performance  evaluation  reports  are  not  to 
be  revised  merely  to  meet  contractor  objections;  rather, 
the  contractor's  comments  are  to  be  attached  to  the 
evaluation  report.  (P-68,  1985,  par.  6-206) 


C.  STANDARD  FORM  1420 

Standard  Form  1420,  PERFORMANCE  EVALUATION-CONSTRUCTION 
CONTRACTS  is  used  for  reporting  contractor  performance.  A 
blank  SF  1420  is  included  as  Appendix  B.  The  form  allows 


space  for  general  contract  data  including:  contractor  name, 
complexity  and  description  of  work,  contract  start  and 
finish  dates,  contract  award  amount,  contract  change  orders, 
and  the  extent  of  subcontracting.  Part  II  allows  the  rater 
to  evaluate  six  performance  elements  (Quality  of  Work,  Time¬ 
liness  of  Performance,  Effectiveness  of  Management,  Compli¬ 
ance  with  Labor  Standards,  Compliance  with  Safety  Standards, 
and  Overall  Performance)  as  either  Outstanding,  Satisfac¬ 
tory,  or  Unsatisfactory.  Any  marks  of  outstanding  or 
unsatisfactory  must  be  supported  by  a  narrative  on  the  back 
of  the  evaluation.  The  evaluations  are  normally  filled  out 
by  the  AROICC  upon  completion  of  the  contract  work. 


D.  THE  ROLE  OF  THE  SMALL  BUSINESS  ADMINISTRATION 

Statistics  provided  by  NAVFAC's  Small  Business  Liaison 

indicate  that  small  business  concerns  receive  approximately 

80%  of  NAVFAC's  contracts  (dollar  value)  each  year. 

Whenever  a  prospective  contractor  is  a  small  business,  the 
Small  Business  Act  vests  in  the  Small  Business  Administra¬ 
tion  sole  authority  to  make  a  determination  of  nonrespon¬ 
sibility.  An  affirmative  responsibility  determination  by 
SBA  is  issued  via  the  medium  of  a  Certificate  of  Competen¬ 
cy  (COC)  .  If  SBA  declines  for  any  reason  to  issue  a 
certificate  of  competency,  the  prospective  contractor 
shall  be  rejected  as  not  responsible.  (P-68,  1985,  par. 

4-407. 4(b)) 

The  Small  Business  Administration  (SBA)  has  this  role  in 
determining  a  contractor's  responsibility: 

(a)  Upon  determining  and  documenting  that  a 
responsive  small  business  lacks  certain  element  of  respon¬ 
sibility  (including,  but  not  limited  to,  competency,  capa¬ 
bility,  capacity,  credit,  integrity,  perseverance,  and 
tenacity) ,  the  contracting  officer  shall-- 


(1)  Withhold  contract  award  (see  19.602-3),  and 

(2)  Refer  the  matter  to  the  cognizant  SBA 
Regional  Office  in  accordance  with  agency  procedures  .  .  . 

(c)  The  referral  shall  consist  of — 

(1)  A  notice  that  a  small  business  concern  has 
been  determined  to  be  nonresponsible ,  specifying  the  ele¬ 
ments  of  responsibility  the  contracting  officer  found 
lacking,  and 

(2)  A  copy  of  the  solicitation,  drawings  and 
specifications,  preaward  survey  findings,  pertinent  tech¬ 
nical  and  financial  information,  abstract  of  bids  (if 
available)  ,  and  any  other  pertinent  information  that  sup¬ 
ports  the  contracting  officer's  determination.  (FAR, 
1984,  par.  19.602-1) 

Once  the  referral  has  been  made  to  the  SBA,  the 
following  procedures  are  followed: 

(a)  Within  15  business  days  (or  a  longer  period 
agreed  to  by  the  SBA  and  the  contracting  agency)  after 
receiving  a  notice  that  a  small  business  concern  lacks 
certain  elements  of  responsibility,  the  SBA  will  take  the 
following  actions: 

(1)  Inform  the  small  business  concern  of  the 
contracting  officer's  determination  and  offer  it  an  oppor¬ 
tunity  to  apply  to  the  SBA  for  a  certificate  of  competency 
(COC)  . 

(2)  Upon  timely  receipt  of  the  application  and 
required  documentation,  send  an  SBA  team  to  visit  the  con¬ 
cern  to  investigate  it  only  for  the  specific  elements  of 
responsibility  that  the  agency  notice  specified  as 
lacking,  and  to  make  recommendations  to  the  SBA  Regional 
Administrator . 

(3)  If  the  Regional  Administrator  plans  to  issue 
or  recommend  issuance  of  a  COC,  provide  advance  notice  of 
the  proposed  action  to  the  contracting  officer  together 
with  a  brief  statement  of  the  reasons  for  it.  If  the  con¬ 
tracting  officer  disagrees  with  the  proposal,  resolve  the 
disagreement  as  provided  in  19.602-3.  (FAR,  1985,  par. 
19.602-2) 


E .  SUMMARY 

This  chapter  described  NAVFAC's  mission,  its  organiza¬ 
tion,  and  its  contracting  procedures.  The  chapter  also 
reviewed  the  regulatory  requirements  for  pre-award  surveys 
and  contractor  performance  evaluations.  Finally,  the 


chapter  discussed  the  Standard  Form  1420  and  the  Small 
Business  Administration's  role  in  determining  a  contractor's 
competency.  The  next  chapter  examines  accepted  theories 
about  control  and  measurement  systems  and  develops  three 
models  of  potential  uses  for  the  contractor  performance 
evaluations.  The  theory  and  models  will  be  used  later 
during  the  analysis  of  research  data. 
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CONTROL  AND  MEASUREMENT  THEORY 


The  contractor  performance  evaluations  provide  a  measure 
of  the  contractor's  performance  on  a  contract.  The 
contractor  performance  evaluation  process  is  intended  to 
control  the  quality  of  construction  contractors  who  receive 
NAVFAC  contracts.  This  chapter  examines  accepted  theories 
of  measurement,  control,  and  evaluation  relevant  to  NAVFAC' s 
use  of  contractor  performance  evaluations. 


CONTROL  THEORY 


Management  control  is  defined  as,  " 


The  process 


through  which  managers  assure  that  actual  activities  conform 
to  planned  activities.  .  .  .  The  control  process  .  .  .  en¬ 
ables  managers  to  detect  deviations  from  the  plan  in  time  to 
take  corrective  action  before  it  is  too  late."  (Stoner, 
1982,  p.  592)  This  definition  implies  that  even  though  one 
is  "in  control,"  actual  performance  can  vary  from  planned. 
A  manager  exerts  control  when  he  takes  action  to  eliminate 
variations  from  plan. 

All  control  systems  have  some  common  characteristics 

.  .  .  any  control  system  has  at  least  these  four 
components : 

1.  An  observation  device  that  detects  or  observes  and 
measures  or  describes  the  activities  or  other  phenomena 
being  controlled.  The  term  for  this  component  may  be 
observer,  detector,  or  sensor. 
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2.  An  assessing  device  that  evaluates  the  performance  of 
an  activity  or  organization,  usually  relative  to  some 
standard  or  expectation  of  what  should  be,  and  identifies 
out-of-control  activities  and  conditions.  The  term  for 
this  component  is  evaluator  assessor,  or  selector. 

3 .  A  behavior  modification  device  for  altering  or  chang¬ 
ing  performance  if  the  need  for  doing  so  is  indicated. 
This  component  may  be  called  a  director,  modifier,  or 
effector. 

4.  A  means  of  transmitting  information  among  the  other 
devices.  This  component's  term  is  communication  network. 
(Anthony,  Deardon,  Beford,  1984,  p.  8,  underlines  added 
for  emphasis) 

This  model  portrays  the  functioning  of  a  control  system 
according  to  Anthony. 


CONTROL  DEVICE 


The  arrows  in  the  model  constitute  the  communication 
network.  In  this  control  system  the  detector,  selector,  and 
effector  functions  could  all  be  performed  by  the  same 
person,  or  each  could  be  performed  by  different  people. 
Within  a  control  system: 

Individuals  typically  occupy  one  of  three  organiza¬ 
tional  positions  with  respect  to  each  information  and 
control  system  they  interact  with: 

1.  The  measured  and  controlled  position.  This  position 
involves  being  measured  by  the  system. 
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2.  The  system  maintenance  position.  Their  primary  task 
is  seeing  that  the  decision  makers  have  the  information 
they  need. 

3.  The  decision  maker  position.  This  position  involves 
receiving  tne  information  from  the  control  system  in  order 
to  make  decisions.  (Lawler  and  Rhode,  1976,  p.  8) 

The  control  system  affects  the  behaviors  of  people  in 
each  position  in  different  ways.  Usually,  the  behavior  of 
greatest  concern  is  that  of  the  person  being  measured. 
Those  who  design  control  systems  should  consider  the 
system's  effects  on  behavior  to  ensure  dysfunctional 
behaviors  are  not  produced. 

All  information  and  control  systems  have  these  charac¬ 
teristics  in  common: 

1.  They  all  have  some  similar  structural  characteristics. 

a.  They  all  collect,  store,  and  transmit  information 
in  the  form  of  abstract  measures  of  reality. 

b.  The  collected  abstract  information  is  stored  and 
transmitted  in  a  specific  form  and  with  specific 
frequency. 

c.  The  summarized  information  is  distributed  to  a 
specific,  usually  pre-determined,  group  of  people. 

2.  All  information  tries  to  accomplish  the  same  thing — 
influence  behavior.  The  crucial  aspect  of  any  control 
system  is  its  influence  on  behavior.  (Lawler  and 
Rhode,  1976,  pp.  5-6) 

Given  that  control  systems  have  some  basic  similarities, 
one  can  ask,  what  qualities  make  one  control  system  better 
than  others?  How  are  good  control  systems  developed  is 
another  good  question.  A  good  place  to  start  is  deciding 
the  purpose  of  the  control  system. 


Some  measure  performance  to  provide  management  with  the 
information  they  need  to  control  employee's  present 
behavior.  Others  provide  top  management  with  the  inform¬ 
ation  they  need  for  long-range  planning.  And  a  third 
group  provides  ongoing  feedback  to  employees  about  their 
job  performance.  (Lawler  and  Rhode,  1976,  p.  33) 


B.  CONTRACTOR  PERFORMANCE  EVALUATIONS  AS  A  CONTROL  SYSTEM 

Lawler  and  Rhode's  previous  quote  described  three 
possible  uses  for  control  systems.  Let  us  now  examine  how 
NAVFAC ' s  Contractor  Performance  Evaluation  system  could 
fulfill  each  of  those  three  uses. 

1 .  Use  I:  Award  Decisions 


Providing  top  management  with  information  for  long- 
range  planning  is  currently  the  actual  purpose  of  NAVFAC ' s 
contractor  performance  evaluation  system.  The  evaluations 
provide  top  management  with  information  which  is  used  to 
make  decisions  about  whether  contractors  receive  future  con¬ 
tract  awards.  This  model  describes  the  control  system: 


CONTROL  SYSTEM 
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In  this  model  the  AROICC  functions  as  the  detector 
and  evaluator.  The  OICC  or  EFD  acts  as  the  effector  because 
the  action  of  interest  is  the  future  contract  award. 

2 .  Use  II:  Controlling  Ongoing  Contractor  Performance 
If  NAVFAC  issued  interim  performance  evaluations  to 
contractors,  then  their  behavior  on  current  contracts  could 
be  affected.  This  model  depicts  the  evaluation  system 
functioning  as  a  control  of  a  contractor's  current  behavior. 

CONTROL  SYSTEM 


AROICC 


Notice  in  this  model  that  the  AROICC  performs  the 
detector,  evaluator,  and  effector  functions  by  himself.  The 
AROICC  is  the  effector  because  the  action  of  interest  is 
controlling  the  ongoing  contractor  performance. 

3 .  Use  III:  Controlling  ArOICC  Performance 

Lawler  and  Rhodes  third  use,  providing  ongoing  feed¬ 
back  to  employees  about  their  job  performance,  could  be 
satisfied  under  NAVFAC ' s  current  evaluation  system. 
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Specifically,  top  management  could  control  the  AROICC's  use 
of  contractor  performance  evaluations  by  rating  the  AROICC ' s 


performance  on:  (1)  the  preparation  of  contractor  evalua¬ 
tions,  and  (2)  the  usefulness  of  contractor  performance 
evaluations  as  a  source  of  information  prior  to  administer¬ 
ing  a  contract.  The  AROICC's  performance  should  improve 
because  preparing  a  better  contractor  performance  evaluation 
will  require  increased  documentation  and  understanding  of 
the  contractor's  performance.  Reviewing  the  contractor's 
past  performance  evaluations  prior  to  administering  a  new 
contract  will  alert  the  AROICC  to  that  contractor's  previous 
problems.  This  two-tiered  model  depicts  that  control  possi¬ 
bility  within  the  NAVFAC  organizational  structure: 


CONTROL  SYSTEM 
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CONTRACTOR 


In  this  model,  on  the  upper  tier  the  OICC  acts  as 
the  detector,  evaluator,  and  effector.  The  OICC  is  the 
effector  because  the  action  of  interest  is  controlling  the 
AROICC's  performance. 

The  three  models  just  presented  include  the  four 
basic  characteristics  of  control  systems  described  by 
Anthony.  The  evaluation  system  also  meets  the  criteria  des¬ 
cribed  earlier  by  Lawler  and  Rhode,  because: 

-  The  evaluations  collect,  store,  and  transmit 
information. 

-  The  information  is  stored  in  a  specific  format  (SF  1420) 
and  is  collected  with  specific  frequency. 

-  The  summarized  information  is  distributed  to  a  specific 
group  of  people  (OICC,  EFD ,  Army  Corps  of  Engineers) . 

-  The  evaluations  purpose  is  to  assist  in  determining  the 
responsibility  of  contractors.  If  one  assumes  contrac¬ 
tors  want  to  be  determined  responsible,  then  the  evalua¬ 
tions  can  be  assumed  to  have  an  influence  on  their 
behavior. 


C.  CONSIDERATIONS  IN  DEVELOPING  AND  EVALUATING  CONTROL 
SYSTEMS 

A  control  system  provides  the  user  with  information  in 
the  form  of  a  measure.  The  following  items  represent  some 
characteristics  which  make  information  useful  to  its  users: 

1.  Usefulness  for  Decision  Making--The  characteristics  of 
information  that  make  it  a  desirable  commodity  can  be 
viewed  as  a  hierarchy  of  qualities,  with  usefulness 
for  decision  making  of  most  importance  ( FASB ,  1980,  p. 
4037).  There  are  many  user-specific  factors  to  con¬ 
sider  when  deciding  what  information  is  most  useful 
for  decision  making.  These  factors  include:  ".  .  . 

the  decisions  to  be  made,  the  methods  of  decision 
making  to  be  used,  and  the  information  already  posses¬ 
sed  or  obtainable  from  other  sources."  (FASB,  1980, 
p.  4037) 
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2.  Understandabil  i  tv--11  In  format  ion  cannot  be  useful  to 
decision  makers  who  cannot  understand  it,  even  though 
it  may  otherwise  be  relevant  to  a  decision  and  be 
reliable."  ( FASB,  1980,  p.  4037) 

3.  Relevance — "Does  the  measure  contain  information  per¬ 
tinent  to  the  decision  at  hand?.  .  .  .  Looking  for¬ 

ward,  relevance  implies  predictive  ability; 

Looking  backward,  relevance  implies  feedback  value." 
(Moses,  1986,  p.  9)  "Relevant  information  is  capable 
of  making  a  difference  in  a  decision  by  helping  users 
to  form  predictions  about  the  outcomes  of  past, 
present,  and  future  events  or  to  confirm  or  correct 
prior  expectations.  Information  can  make  a  difference 
to  decisions  by  improving  decision  makers'  capacities 
to  predict  or  by  providing  feedback  on  earlier  expec¬ 
tations.  Without  a  knowledge  of  the  past,  the  basis 
for  a  prediction  will  usually  be  lacking."  (FASB, 
1980,  p.  4038) 

4.  Timeliness — "If  information  is  not  available  when  it 

is  needed  or  becomes  available  so  long  after  the 
reported  events  that  it  has  no  value  for  future 
action,  it  lacks  relevance  and  is  of  little  or  no 
use."  (FASB,  1980,  p.  4038)  "Timeliness  does  not 

cause  relevance,  but  lack  of  timeliness  can  make  an 
otherwise  useful  measure  irrelevant."  (Moses,  1986, 
p.  10) 

5.  Reliability — "To  what  degree  do  repeated  measurements 

of  the  same  attribute  vary?"  (Euske,  1984,  p.  86) 

"Reliability  connotes  stability,  consistency,  objec¬ 
tivity,  dependability,  and  absence  of  bias.  In  short 
reliability  is  the  relative  absence  of  errors  of 
measurement  in  the  measurement  instrument."  (Moses, 
1986,  p.  5) 


Requisite  Variety — "Do  the  measures  make  sufficiently 
fine  distinctions  among  objects?  A  given  decision 
situation  may  require  a  particular  kind  of  separation 
or  differentiation  among  objects.  If  a  measurement 
process  is  to  provide  useful  information  for  the  deci¬ 
sion,  it  should  result  in  measures  that  capture  the 
required  distinctions."  (Moses,  1986,  p.  12) 


Completeness- -"A  crucial  aspect  of  any  control  system 
is  how  completely  and  inclusively  it  measures  the 
behaviors  that  need  to  be  performed  by  a  job  holder. 

If  measures  are  incomplete,  a  person  will  be 
motivated  to  perform  only  a  portion  of  the  behavior 
needed  for  organizational  effectiveness."  (Lawler  and 
Rhode,  1976,  p.  42) 
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8.  Valence — "To  what  degree  are  the  measurements  tied  to 
the  organizational  reward  system?  .  .  .  Individuals 
act  to  satisfy  personal  needs.  To  the  extent  that 
some  needs  are  satisfied  by  the  organizational  reward 
system,  and  to  the  extent  that  rewards  are  linked  to 
specific  measurements,  behavior  to  influence  the 
measurements  will  be  motivated."  (Moses,  1986,  p.  16) 

9.  Control lability--" Can  the  person  measured  influence 
the  events  or  objects  that  are  represented  by  the 
measurements?  Without  control,  individuals  cannot 
feel  responsible  for  outcomes  or  feel  satisfaction  in 
outcomes,  and  c  msequently,  will  not  be  intrinsically 
motivated  to  perform.  Without  control  individuals 
will  see  no  link  between  his  or  her  behavior  and 
potential  rewards,  and  consequently  will  not  be 
extrinsically  motivated."  (Moses,  1986,  pp.  16-17) 

10.  Objectivity — "Some  systems  use  objective  measures 
while  others  use  subjective  ones.  .  .  .  Research 
shows  that  the  more  objective  measures  are,  the  more 
likely  they  are  to  motivate  behavior.  ...  It  is 
only  logical  that  if  individuals  don't  understand  how 
the  measure  operates  because  it  is  highly  subjective, 
they  will  see  little  connection  between  their  behavior 
and  any  reward  based  on  the  measure.  .  .  .  Another 
reason  objectivity  of  measures  is  important  has  to  do 
with  the  climate  of  mistrust  present  in  most  organiza¬ 
tions.  When  employees  do  not  believe  they  will  be 
evaluated  fairly,  they  do  not  believe  that  good  per¬ 
formance  on  their  part  will  lead  to  rewards,  and  as  a 
result  motivation  is  low."  (Lawler  and  Rhode,  1976, 
pp.  42-47) 


D.  OTHER  CONSIDERATIONS  FOR  DESIGNING  A  CONTROL  SYSTEM 

Once  the  purpose  for  a  system  is  made  clear,  and  all  the 
user-specific  needs  are  identified,  the  system  designer  can 
proceed.  These  other  factors  should  be  considered  when 
developing  a  control  system.  Following  each  listed  factor 
below  is  a  description  of  that  factor,  and  a  discussion  of 
what  makes  each  factor  a  desirable  feature  of  a  control 
system . 
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1.  Nature  of  Standards--The  research  and  theory  on 
extrinsic  motivation  make  one  point  quite  clear: 
Employees  are  not  motivated  to  reach  goals  that  they 
don't  feel  they  can  achieve.  .  .  .  The  research  on 
goal  setting  (e.g.,  Locke,  1968)  suggests  that  indi¬ 
viduals  tend  to  achieve  easy  goals  but  not  exceed 
them.  The  result  is  mediocre  performance.  .  .  To  set 
goals  at  the  proper  level  of  difficulty,  individuals 
who  have  information  on  where  the  standards  should  be 
set  must  be  involved.  Further,  the  individuals  whose 
performance  is  being  measured  must  be  aware  of  the 
information  that  was  used  to  set  the  standards  so  they 
will  realize  that  the  standards  are  reasonable. 

2.  Source  of  Discrimination — As  with  standard  setting,  it 
is  crucial  who  acts  as  discriminator.  .  .  .  Whoever 
it  is  must  have  two  attributes:  the  knowledge  to  make 
the  comparisons  and  the  trust  of  the  person  being 
measured. 

3 .  Pattern  of  Communicat ion--Control  systems  vary  widely 
in  terms  of  who  receives  information  about  deviations 
from  the  standard.  .  .  .  For  extrinsic  motivation  to 
be  present,  the  person  who  has  the  power  to  give 
rewards  must  receive  the  information  about  the  results 
of  the  discrimination.  This  is  a  precondition  to 
rewards  being  allocated  on  the  basis  of  performance 
which  in  turn  is  a  precondition  to  the  perception  that 
rewards  are  based  upon  performance.  However,  for  this 
perception  to  exist,  ...  it  is  desirable  to  have  the 
person  whose  performance  is  being  measured,  and  other 
employees  who  are  doing  similar  work,  receive  the 
communication . 

4 .  Speed  and  Frequency  of  Communication--Control  systems 
differ  in  how  quickly  they  report  the  results  of  their 
measurement  process.  ...  As  a  general  rule,  commun¬ 
ications  about  performance  measures  are  most  effective 
in  producing  extrinsic  motivation  when  they  are  fast 
and  frequent.  When  the  communications  about  perform¬ 
ance  are  delayed,  it  is  impossible  to  closely  tie 
rewards  to  the  actual  performance  of  the  person  and 
this  has  the  effect  of  reducing  the  perceived  rela¬ 
tionship  between  performance  and  rewards.  It  also 
creates  doubts  among  employees  about  the  validity  and 
usefulness  of  the  performance  measurement  data. 
(Lawler  and  Rhode,  1976,  pp.  42-43) 

The  theory  on  performance  appraisals  offers  this  advice: 

Today's  employees  want  more  from  an  organization  than 

just  their  salary.  They  want  to  know  how  they  are 


performing  in  their  jobs.  .  .  .  People  do  not  mind  being 
evaluated  if  the  appraisal  system  is  fair  and  comprehen¬ 
sive  to  them.  (Cohen  and  Jaffee,  1982,  p.  94) 


Wells  says  an  effective  and  defensible  performance 
appraisal  system  should  have  these  qualities: 

1.  Understandable  Standards--Everv  employee  should  under¬ 
stand  the  standards  against  which  he  or  she  will  be 
evaluated.  .  .  .  They  should  have  prior  knowledge  of 
what  is  expected. 

2.  Knowledgeable  Appraisers--Another  problem  with  similar 
implications,  may  arise  when  immediate  superiors  do 
not  possess  sufficient  technical  knowledge  or 
expertise  in  an  area  of  specialization  to  reasonably 
judge  the  results  produced  by  others. 

3 .  Appraisers  Should  Be  Well  Trained — There  are  a  number 
of  reasons  why  it  is  critical  that  those  conducting 
the  appraisals  should  be  well  trained  to  guarantee 
consistency,  .  .  .  Consistency  in  administration  is 
integrally  linked  to  the  defensibility  of  the  system 
in  the  event  of  a  legal  challenge. 

4 .  Communication  of  Policy  and  Purpose — The  purposes  and 
uses  of  appraisals  should  be  clearly  stated  in  the 
policy  manual,  as  well  as  in  the  employee  handbook.  . 
.  .  Fully  informing  all  employees  of  appraisal  policy 
and  purposes  will  minimize  uncertainty  and  potential 
resistance.  Employees  understanding  will  increase  the 
potential  for  positive  results  from  appraisals. 
(Wells,  1982,  pp.  777-781) 

The  items  Wells  stresses  are  consistent  with  Lawler  and 
Rhodes  considerations  for  control  systems.  The  theories  and 
models  presented  in  this  chapter  will  be  used  in  Chapter  V 
during  the  analysis  of  NAVFAC's  current  use  of  performance 
evaluations.  The  next  chapter  presents  the  data  generated 
during  the  research  about  NAVFAC's  current  use  of  contractor 
performance  evaluations. 
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IV.  RESEARCH  DATA 


A.  OVERVIEW 

This  chapter  describes  the  research  methods  used  to 
examine  NAVFAC ' s  current  use  of  Contractor  Performance 
evaluations.  The  research  data  produced  is  also  presented. 
The  following  sources  of  data  were  used: 

1.  Mail  survey  of  AROICC's. 

2.  Mail  survey  of  construction  contractors. 

3.  Telephone  survey  of  field  office  contract  specialists. 

4.  Mail  requests  for  information  from  EFD  points  of 
contact . 

5.  Mail  survey  of  field  office  Supervisory  Civil 
Engineers. 

6.  Small  Business  Administration  sources. 

The  mail  and  telephone  surveys  were  prepared  using 
Dillmans'  Total  Design  Method  (Dillman,  1982).  The  Total 
Design  Method  is  a  book  which  provides  comprehensive 
instructions  on  how  to  prepare  successful  mail  and  telephone 
surveys.  Among  other  things,  the  Total  Design  Method:  (1) 

helped  reduce  evaluation  bias,  (2)  gave  tips  for  increasing 
the  survey  response  rate,  and  (3)  provided  examples  of 
properly  phrased  survey  questions.  The  purpose  for,  back¬ 
ground  about,  and  survey  results  for  each  data  source  are 
presented  below. 


B.  AROICC  SURVEY 


1 .  Purpose 

A  mail  survey  of  AROTCC's  was  used  to  determine: 

-  AROICC  opinions  about  the  effectiveness  and  usefulness 
of  the  contractor  performance  evaluations. 

-  The  amount  of  training  AROICC' s  have  in  completing  an 
evaluation . 

-  AROICC  perceptions  about  management's  emphasis  on 

evaluations . 

-  AROICC  opinions  on  the  use  of  interim  evaluations. 

2 .  Background 

Before  the  AROICC  survey  form  was  finalized,  8 
"pilot"  surveys  were  sent  to  AROICC 's  to  determine  if  the 
responses  would  generate  acceptable  data.  All  8  surveys 
were  returned.  A  review  of  the  responses  resulted  in  the 
elimination  of  one  question,  and  minor  rewording  of  two 
others.  The  final  survey  (Appendix  C)  was  sent  to  60 
AROICC' s.  The  AROICC 's  surveyed  were  chosen  from  NAVFAC ' s 
P-1,  Navy  Civil  Engineer  Corps  Directory.  Only  LTJG's  and 
above  with  more  than  1  year  experience  as  an  AROICC  were 
surveyed.  54  of  the  60  (90%)  surveys  were  returned.  The 
data  summary  below  includes  the  8  pilot  surveys,  for  a  final 
response  from  62  of  68  (91%)  AROICC's. 

3 .  AROICC  Survey  Results 

The  AROICC  survey  questions  and  summarized  responses 


are  presented  below. 


Question  1.  Approximately  how  many  satisfactory,  outstand¬ 
ing,  and  unsatisfactory  contractor  performance  evaluations 
have  you  written  during  your  career  as  a  contract 
administrator? 

Totals:  1361  (86%)  --Satisfactory 

119  ( 7 . 6% ) --Outstanding 
100  (6.4%) — Unsatisfactory 
1580 

The  average  respondent  has  filled  out  25.4  evaluations 
each . 

Question  2.  Does  your  EFD  have  an  instruction  governing 
contractor  performance  evaluations?  (All  EFD's  do  have 
instructions . ) 


45  of  62  (64%) — YES 

16  of  62  (26%) — NO 
1  of  62 — Did  not  answer 


Question  3.  If  answer  to  question  2  was  yes,  how  much  help 
does  the  instruction  provide  when  you  are  filling  out  an 


evaluation? 


1  of  45 — A  Big  Help  X 

22  of  45 — Some  Help  X 

16  of  45 — A  Little  Help  X 
4  of  45 — No  Help  X 


2  of  45 — Did  not  answer 


3  points  =  03 
2  points  =  44 
1  point  =  16 
0  points  =  __0 

65  points/43 
respondents  = 
1.51 


The  weighted  average  response  of  1.51  falls  mid-way  between 
a  little  and  some  help. 


Question  4.  Have  you  ever  received  any  training  on  how  to 
complete  the  performance  evaluation  form? 


23  of  62  (38.1%) — YES 
39  of  62  (61.9%) — NO 


If  YES,  which  of  these  statements  most  accurately 
describes  the  level  of  your  training? 

4- -Formal  Classroom  training 

5- -Briefed  during  AROICC  staff  meeting 

11 — OJT  by  Contract  Specialist  or  another  AROICC 
i_2--Given  a  copy  of  an  old  evaluation  as  a  guide 
32* 


*--Some  of  the  respondents  listed  more  than  one 
type  of  training 


Question  5.  Do  you  ever  review  a  contractor's  record  of 
past  performance  prior  to  the  start  of  a  new  contract? 


23  of  62  (38.1%) — YES 

39  of  62  (61.9%) — NO 

If  yes,  approximately  how  often  do  you  do  so? 

12  do  so  more  than  50%  of  the  time 

11  do  so  less  than  50%  of  the  time 

Question  6.  Do  you  know  where  you  could  find  a  file  of  all 
of  a  particular  contractor's  past  performance  evaluations? 

25  of  62  (40%) — YES 

37  of  62  (60%) — NO 

Question  7.  On  a  scale  of  1  to  10,  how  effective  do  you 
think  the  existing  performance  evaluation  system  is  at 
improving  the  overall  performance  of  NAVFAC  contractors?  (1 
not  effective  to  10  very  effective) 

not  effective  very  effective 

scale — 0  123456789  10 
#  of  — 1  11  11  20  7  3  2  3  0  0  1 
responses 

The  average  was  3.03.  A  trimmed  average  (omit  the 
lowest  and  highest  response)  was  2.96.  73%  of  the  responses 

were  3  or  less. 

Question  8 .  Have  you  ever  written  an  interim  evaluation  on 
a  contractor  mid-way  through  a  project? 

15  of  62  (24%) — YES 
47  of  62  (76%) — NO 

If  yes,  what  effect  does  the  interim  evaluation  have  on  a 
contractor's  performance? 

10  of  15  said  there  was  a  noticeable  improvement  in  the 
contractor's  performance. 

Question  9.  Do  you  think  interim  evaluations  would  help  you 
to  positively  influence  a  contractor's  behavior? 

41  of  62  (67%) — YES 
18  of  62  (29%) — NO 
3  of  62  (  4%) --Did  not  answer 


.Y.'/.V 


i 


Why? 


The  YESES  typically  said: 

a.  Contractor  will  adjust  behavior  to  please  the 
NAVY 

b.  Early  feedback  will  prove  useful  to  the 
contractor 

c.  Will  provide  an  evaluation  on  file  if  the 
AROICC  is  transferred 

d.  Contractors  are  concerned  about  their 

reputations 


The  NO's  typically  said: 

a.  Contractors  are  only  motivated  by  money 

Question  10.  Which  of  these  statements  most  accurately 
describes  how  much  emphasis  your  office  management  places  on 
the  performance  evaluations? 


18 

of 

62 

19 

of 

62 

14 

of 

62 

5 

of 

62 

5 

of 

62 

1 

of 

62 

Ouestion 

11- 

evaluate 

our 

(29%) — Evaluations  are  stressed  frequently 
(31%) --Evaluations  are  stressed  occasionally 
(22%) — Evaluations  are  stressed  if  contractor 
performance  unsatisfactory 
(  8%) — Evaluations  are  stressed  when  overdue  to 
the  EFD 

(  8%) — Evaluations  are  never  stressed 
(  2%) — Did  not  answer 


What 


comments  do  you  have  about  the  way  we 
Please  include  any  suggestions 


for  improving  the  evaluation  process. 

This  is  a  summary  of  the  comments  received. 


Frequency 

13 


11 


6 


5 


Comment 


The  evaluations  are  not  effective  because 
NAVFAC  issues  subsequent  awards  to  contractors 
who've  gotten  unsatisfactory  evaluations 

The  evaluation  forms  need  more  performance  and 
rating  categories  to  allow  for  a  more  detailed 
evaluation  of  the  contractor's  performance 

The  EFD's  need  to  issue  periodic  summaries  of 
contractor  performance  for  use  by  field 
offices 

Unsatisfactory  evaluations  are  avoided  because 
they  take  too  much  time,  effort,  and  documen¬ 
tation  to  support 


3 


The  evaluations  are  too  subjective 


3  A  policy  should  be  established  to  mandate 

denial  of  awards  after  a  certain  number  of 
unsatisf,  ctory  evals 

3  We  need  meaningful  rewards  for  outstanding 

contractors 

2  The  Small  Business  Administration  rarely  rules 

against  a  contractor  in  a  Certificate  of 
Competency  case 

2  Interim  evaluations  should  be  sent  to 

contractors 

2  AROICC's  should  be  better  trained  in  filling 

out  evaluations 

2  Time  constraints  affect  the  quality  of 

evaluations 

1  The  narrative  should  be  written  in  bullet 

statements 

1  Need  some  criteria  for  translating  separate 

ratings  on  each  item  into  an  overall  rating 

1  A  contractor's  performance  is  affected  by  the 

quality  of  the  project  design 

1  Copies  of  the  evaluations  should  be  sent  to 

the  contractors'  bonding  companies 


C .  CONTRACTOR  SURVEY 
1 .  Purpose 

The  contractor  survey  was  used  to  determine: 

-  Contractor  opinions  about  the  quality  of  NAVFAC's 
performance  evaluations. 

-  Contractor  opinions  about  the  performance  evaluation 
process  and  how  it  might  be  improved. 

-  Contractor  systems  for  evaluating  subcontractors 

-  Contractor  opinions  about  receiving  interim  evaluations. 


A, 
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2 .  Background 

Time  constraints  prevented  the  use  of  a  "pilot" 
survey.  Contractor  mailing  lists  with  contractor  names, 
addresses,  and  points  of  contact  were  requested  from  each 
EFD.  Contractors  on  the  lists  were  to  have  regularly  per¬ 
formed  contracts  within  the  EFD.  4  of  6  EFD's  provided  the 
contractor  lists,  but  only  2  of  4  included  names  of 
contractor  points  of  contact.  Using  those  lists,  surveys 
were  sent  to  80  contractors.  32  of  the  80  (40%)  returned 
the  surveys,  but  13  did  not  complete  them  because  they  had 
never  seen  nor  received  a  performance  evaluation  from 
NAVFAC .  Of  the  19  who  did  complete  the  surveys,  10  contrac¬ 
tors  had  seen  or  received  an  evaluation,  and  9  had  not.  A 
copy  of  the  contractor  survey  is  included  as  Appendix  D. 

3 •  Contractor  Survey  Results 

The  contractor  survey  questions  and  summarized 
responses  are  presented  below. 

Question  1.  On  a  scale  of  1  (no  value)  to  10  (high  value) , 
how  much  value  do  you  assign  to  the  performance  evaluation 
rating  you  receive  on  a  project? 

no  value  high  value 
scale — 1  23456789  10 
#  of  — 4  00030143  4 
responses 

The  overall  average  was  6.6.  The  average  for  thore  who  have 
seen  an  evaluation  before  was  7.9.  63%  of  the  responses 
were  7  or  above. 
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Question  2.  How  useful  to  you  think  interim  evaluations 
would  be  in  providing  you  feedback  about  your  performance  on 
a  contract? 


9  of  19 
4  of  19 
3  of  19 
2  of  19 
1  of  19 


(47%) --Very  Useful 
( 2 1% ) --Fairly  Useful 
( 16% ) — Useful 
(11%) — Of  Little  Use 
(  5%) --Did  Not  Answer 


Of  the  10  who1  ve  seen  an  evaluation  before,  9  of  10  said 
fairly  or  very  useful. 

Question  3.  Would  you  like  to  see  NAVFAC  issue  interim 
performance  evaluations  on  its  contracts? 

13  of  19  (68%) — YES 

6  of  19  (32%) — NO 

Why?  The  yes's  said: 

a.  It  would  eliminate  any  surprises  in  the  final 
evaluation 

b.  It  would  improve  AROICC/contractor  communications 

c.  Contractors  need  feedback  to  know  if  work  satisfies 
the  government 

d.  Contractors  can  correct  mistakes  before  it  is  too 
late 

The  no's  said: 

a.  AROICC's  are  not  qualified  to  make  the  evaluations 

b.  Contractors  already  know  how  good  their  performance 
is . 


Question  4.  Do  you  have  a  system  for  evaluating  your 
subcontractors? 


None  of  the  respondents  have  a  formal  evaluation  system. 
All  make  basic  subjective  judgments  about  the  subcontrac¬ 
tor  performance.  Those  who  don't  perform  well  are  not 
used  on  future  jobs. 

Question  5.  Would  you  find  it  useful  to  receive  a  copy  of 
your  performance  evaluation  for  each  contract  you  perform? 

18  of  19  (95%) — YES 
1  of  19  (  5%) — NO 

Comments :  These  general  comments  were  made  by  contractors 

who  were  surveyed: 


•  < 
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Frequency 


Comment 


2  Evaluations  are  improperly  used  by  AROICC's  as 

a  bargaining  tool  at  contract  close  out 

2  Evaluations  are  too  subjective 

2  Evaluations  are  poorly  documented 

1  Contractors  should  also  evaluate  the  AROICC's 

1  An  appeal  process  is  needed  to  mediate 

disputed  evaluations 

1  Evaluations  don't  matter  because  poor 

contractors  still  receive  contract  awards. 

D.  CONTRACT  SPECIALIST  SURVEY 

1 .  Purpose 

A  telephone  survey  of  contract  specialists  in  20  of 
NAVFAC ' s  field  contract  offices  was  used  to  determine: 

-  How  performance  evaluations  were  used  in  support  of  the 
pre-award  survey  process. 

-  If  the  evaluations  provided  contract  specialists  enough 
information  to  determine  contractor  responsibility. 

-  Contract  specialists  attitudes  about  the  level  of 
management  emphasis  on  performance  evaluations. 

-  Contract  specialists'  use  of  the  EFD  performance 
evaluation  files. 

2 .  Background 

Survey  questions  were  asked  uniformly  to  all  20 
respondents,  and  answers  were  consistently  recorded.  5  of 
the  20  respondents  were  not  capable  of  answering  some  of  the 
questions  because  of  their  offices'  unique  organizational 
structure.  This  information  is  used  with  the  caveats  that: 
(1)  the  survey  population  is  statistically  quite  small,  and 
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(2)  the  size  and  volume  of  contracts  administered  in  each 
surveyed  office  was  not  determined.  A  copy  of  the  survey  is 
included  as  Appendix  E. 


3 .  Contract  Specialist  Survey  Results 

The  contract  specialist  survey  questions  and  summar¬ 
ized  responses  are  presented  below. 


Question  1.  What  percent  of  the  time  do  you  use  a 
contractor's  past  performance  evaluations  to  assist  you 
during  the  Pre-Award  Survey  process? 


1  of  20  (  5%) --Use  the  evaluations  100%  of  the  time 
8  of  20  (40%) --Use  the  evaluations  if  the  contractor 
has  done  work  in  their  office  before 
11  of  20  (55%) — Use  the  evaluations  less  than  10%  of  the 

time.  They  prefer  to  rely  on  phone 
calls  to  contractor  provided  references 
for  information  about  a  contractor's 
past  performance. 


Question  2 .  What  percent  of  the  time  do  you  discuss  the 
pre-award  survey  results  with  the  AROICC  assigned  to  the 
project? 


1  of  20 
4  of  20 
11  of  20 


4  of  20 


(  5%) — The  AROICC  does  the  pre-award  survey 
(20%)— Notify  the  AROICC  90%  of  the  time 
(55%) — Notify  the  AROICC  if  the  contractor's 
past  performance  is  less  than 
satisfactory 

(20%) — Do  not  notify  the  AROICC  at  all 


Question  3.  Other  than  your  own  files,  do  you  use  any  other 
sources  of  contractor  performance  evaluations? 


14 

of 

20 

(70%) 

— Use  no  other 

source 

5 

of 

20 

(25%) 

--Use  their  EFD 
time 

's  file  less 

than  5%  of  the 

1 

of 

20 

(  5%) 

— Does  not  even 
evaluations 

keep  its  own 

file  of 
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Question  4.  Do  the  performance  evaluations  give  you  enough 
information  to  do  the  pre-award  surveys  properly? 

8  of  20  (40%) — Said  the  information  was  adequate 
7  of  20  (35%) --Said  the  information  on  the  evaluations 
was  too  general,  and  that  more  specific 
comments  about  contractor  performance 
should  be  made 

5  of  20  (25%) --Provided  an  inadequate  response 

Question  5.  How  much  management  emphasis  is  placed  cn 
performance  evaluations  within  your  office? 

4  of  20  ( 20% ) --Strong  management  emphasis 

7  of  20  (35%) --A  fair  amount  of  management  emphasis 

5  of  20  (25%) --A  little  management  emphasis 

4  of  20  (20%) — Provided  an  inadequate  response 

Question  6.  In  the  past  year  how  many  contractors  did  your 
office  withhold  awards  from  because  of  an  unsatisfactory 
record  of  past  performance? 

12  of  20  (60%) — Zero 
5  of  20  (25%) — One 
2  of  20  (10%) — Four 
1  of  20  (  5%) — Six 


E.  EFD  INFORMATION 

The  following  information  was  gathered  from  the  EFD 
points  of  contact  in  response  to  mail  and  telephone 
requests.  1  of  the  6  EFD's  chose  not  to  provide  any 
information. 

1.  All  5  EFD's  have  instructions  governing  the  use  and 
preparation  of  contractor  performance  evaluations. 

2.  2  of  5  EFD  instructions  encourage  the  use  of  interim 
evaluations  if  the  contractor's  performance  is 
unsatisfactory . 

3.  All  EFD's  keep  past  performance  evaluations  separated 
by  contractor. 

4.  Pacific  Division  augments  the  SF1420  with  its  own 
"Supplemental  Information  Sheet"  (Appendix  F)  .  This 
sheet  allows  AROICC's  to  rate  contractors  on  more 
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specific  items.  The  sheet  also  has  two  extra  ratings, 
marginal  and  exceptional. 

5.  Pacific  Division  keeps  an  evaluation  record  form 
(Appendix  G)  on  each  contractor.  That  form  summarizes 
a  contractor's  performance  evaluations  from  many 
contracts . 

6.  Northern  Division  is  trying  to  implement  a  computer 
data  base  which  will  create  a  separate  file  of  evalua¬ 
tion  data  for  each  contractor.  The  data  base  will  be 
accessible  to  all  of  Northern  Divisions  field  offices. 


F.  SUPERVISORY  CIVIL  ENGINEER  SURVEY 

1 .  Purpose  and  Background 

A  mail  survey  of  10  supervisory  civil  engineers  was 
conducted  to  assess  their  attitudes  about  contractor  per¬ 
formance  evaluations.  7  responses  were  received  (70%).  A 
copy  of  the  survey  is  included  as  Appendix  H.  Several  of 
the  survey  questions  were  not  well  prepared  and  their 
responses  were  unsuitable  for  use.  This  information  is  used 
with  the  caveat  that  the  sample  population  is  statistically 
very  small. 

2 .  Supervisory  Civil  Engineer  Survey  Results 

The  supervisory  civil  engineer  survey  questions  and 

summarized  responses  are  presented  below.  Those  questions 

which  were  inadequately  prepared  are  not  included  here. 

Question  3.  Do  you  provide  your  AROICC's  any  training  on 
how  to  complete  the  performance  evaluation  form? 

3  of  7  said  YES 

3  of  7  said  NO 

1  of  7  does  all  evaluations  himself, 
training  needed 


therefore  no 


Question  4.  Does  your  office  keep  separate  performance 
evaluation  files  on  each  contractor? 

5  of  7  said  NO 
2  of  7  said  YES 

Question  6.  Does  your  office  write  interim  evaluations  on 
contractors  midway  through  a  report? 

5  of  7  said  NO 
2  of  7  said  YES 

Question  7.  Do  you  think  NAVFAC  should  issue  interim  evalu¬ 
ations  on  all  of  its  construction  contracts? 

6  of  7  said  NO — They  fear  the  field  office  staff  already 

has  more  work  than  it  can  comfortably 
handle 

1  of  7  said  YES 

Question  9.  If  you  had  access  using  the  office  computer  to 
a  file  listing  all  of  a  contractor's  past  performance 
evaluations,  would  you  use  it  to  get  a  feel  for  how  a  pros¬ 
pective  contractor  of  yours  usually  performs? 

6  of  7  said  YES 

1  of  7  said  NO 

Question  10.  Do  you  think  we  should  give  our  contractors  a 
copy  of  their  performance  evaluations  at  the  end  of  a 
contract? 

5  of  7  said  YES 

2  of  7  said  NO 

Question  11  asked  for  comments  on  how  NAVFAC  could  improve 
its  evaluation  system.  These  responses  were  received: 

Frequency  Comment 

2  The  evaluations  should  include  more  specific 

items,  and  have  more  rating  categories 

2  Evaluations  have  no  effect  on  contractors 

because  the  poor  performers  continue  to  get 
new  contract  awards 


1 


The  excessive  amount  of  documentation  needed 
to  support  an  unsatisfactory  rating  is 
burdensome 


The  form  should  require  an  explanation  of  a 
contractor's  satisfactory  performance  also 

Evaluations  are  given  superficial  treatment  by 
AROICC's  because  it  takes  too  much  time  to 
prepare  a  good  evaluation 


G.  SMALL  BUSINESS  ADMINISTRATION  DATA 

1 .  Purpose  and  Background 

When  small  businesses  are  involved  the  Small 
Business  Administration  has  the  authority  to  override  a 
NAVFAC  decision  of  contractor  non-responsibility.  That 
override  occurs  when  a  Certificate  of  Competency  is  issued. 
2  of  the  AROICC's  responding  to  the  survey  said  the  SBA 
rarely  rules  against  contractors  on  COC  decisions.  The  data 
presented  below  comes  from  the  General  Accounting  Office 
report  #RCED-86-120BR  dated  April  1986  and  titled  "Small 
Business  Administration-Status,  Operations,  and  Views  of  the 
Certificate  of  Competency  Program."  The  report  highlights 
actual  uses  of  the  contractor  performance  evaluations  within 
the  SBA's  COC  system. 

2 .  SBA  Data 

All  data  presented  below  comes  from  the  above  cited 
GAO  report.  The  data  represents  all  types  of  contracts 
reviewed  by  the  SBA  including  some  construction  contracts. 


aw  a 


a.  This  table  provides  statistics  on  the  COC  activity  for 
1981-1985. 


Fiscal  Refer-  Contractor  COC  Applica- 

Year  rals  Applica-  approv-  tion 

to  SBA  tions  als  denials 


Direct 
Awards 
without 
a  COC 


Totals  15733 


5700 


2971 


2042 


The  numbers  of  approvals,  direct  awards,  and  denials  do  not 
equal  the  number  of  applications  because  of  such  things  as 
applications  withdrawn,  referrals  withdrawn,  and  procure¬ 
ments  withdrawn. 

Based  on  the  total  figures  for  the  five  years,  there  were 
10,033  contractors  who  did  not  apply  for  COC's,  and  another 
2042  who  were  denied  COC's.  Those  12,075  contractors  repre¬ 
sent  76%  of  the  total  referrals  to  the  SBA.  Thus  76%  of 
contractors  referred  to  the  SBA  do  not  receive  the  contract 
award . 

b.  The  GAO  report  included  a  survey  of  279  cases.  The 
survey  looked  at  the  processing  time  for  those  cases.  The 
FAR  allows  the  SBA  15  days  to  process  a  case,  or  a  longer 
time  if  agreed  to  by  the  referring  contracting  agency.  The 
results  of  the  survey  showed: 


Number  of 
Workdays 


Cases 

Number  Percent 


15  or  less 
16-20 
21-25 
26-35 

36  or  more 


Total 


44 . 6 
29 . 3 
11. 1 
5.9 
6. 3 
100 


Though  only  44.6%  of  the  cases  were  completed  within  the 
allowed  15  days,  85%  were  completed  within  25  days. 

c.  The  SBA's  contractor  review  procedures  are  generally 
consistent  with  DOD's  and  GSA 1 s  review  procedures. 


d.  No  statutory  or  regulatory  criteria  exist  to  guide  SBA 
in  evaluating  prior  performance  when  making  COC  decisions. 
In  addition,  SBA  has  not  established  specific  written 
criteria  to  make  such  determinations.  The  SBA  evaluators 
must  rely  on  professional  judgment  much  like  the  AROICC's 
do . 

e.  Of  287  cases  sampled,  109  were  referred  to  the  SBA 
because  of  the  contractors  poor  prior  performance.  Of  those 
109,  SBA  agreed  with  the  referring  agency  in  51  cases,  or 
about  47%,  and  denied  the  COC. 


H.  SUMMARY 

This  chapter  introduced  data  from  several  sources  about 
NAVFAC's  current  use  of  the  contractor  performance  evalua¬ 
tions.  The  next  chapter  provides  an  analysis  of  that  data 
from  the  viewpoint  of  accepted  control  and  measurement 
theory  from  Chapter  III. 


ANALYSIS  OF  CONTRACTOR  EVALUATIONS  AS  A  CONTROL  MECHANISM 

Chapter  III  presented  some  accepted  control  and  measure¬ 
ment  theory  as  well  as  three  possible  uses  for  the  contrac¬ 
tor  performance  evaluations.  Chapter  IV  presented  data 
about  NAVFAC's  current  uses  of  the  contractor  performance 
evaluations.  This  chapter  provides  an  analysis  of  those 
current  uses  from  the  viewpoint  of  the  accepted  theories. 
The  analysis  focuses  on  the  three  possible  uses  for  the 
evaluations  from  Chapter  III. 

A.  USE  I — AWARD  DECISIONS 

The  FAR,  DFARS ,  and  NAVFAC  P-68  each  cite  only  one 
purpose  for  the  contractor  performance  evaluations;  to 
provide  information  about  contractor  past  performance  during 
the  pre-award  survey.  This  control  model  reproduced  from 
Chapter  III  depicts  the  control  system. 

CONTROL  SYSTEM 

^  AROICC  "^OICC  i 

i  ) 

!  DETECTOR - i  EVALUATOR  -  j  -‘EFFECTOR 

r  - - - 

\ 

observes  award 

performance  ./  decisions 

/ 

\  / 

IjS  ONTRACTOR  ^ 
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2 .  Analysis 

When  analyzed  f  rcm.  the  viewpoint  of  accepted  control 
theory,  NAVFAC 1 s  current  use  of  the  contractor  performance 
evaluation  system  for  making  contra  -t  award  decisions  is 
weak  in  the  following  areas. 

a.  Standards 

Lawler  and  Rhode ,  ana  We..s  sa  i  u  control  and 

appraisal  systems  should  have  well-defined  standards.  The 
standards  should  be  understood,  and  should  not  be  too  easy 

or  too  difficult  to  attain.  The  FAR,  DOD  FAR  Supplement, 
and  NAVFAC  P-68  all  are  silent  about  standards  for  contrac¬ 
tor  performance.  No  guidance  exists  which  defines  what  con¬ 
stitutes  satisfactory,  unsatisfactory,  or  outstanding 
performance.  Instead,  each  AROICC  is  left  to  exercise  his 
own  professional  judgment  when  evaluating  a  contractor's 
performance . 

b.  Usefulness  For  Decision  Making 

This  characteristic  of  information  is  most 
important.  Despite  its  importance,  11  of  20  contract 
specialists  use  the  evaluations  less  than  10%  of  the  time 
when  doing  pre-award  surveys,  and  14  of  20  do  not  use  other 
available  sources  of  evaluations.  5  of  7  supervisory  civil 
engineers  said  their  offices  don't  keep  separate  files  of 
performance  evaluations.  These  facts  indicate  NAVFAC  could 
improve  its  use  of  the  contractor  performance  evaluations  in 
the  contract  award  decision  making  process. 
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The  following  association  of  responses  by  the 
contract  specialists  provides  some  insight  into  the  useful¬ 


ness  of  the  performance  evaluations.  The  responses  to 
question  1  (What  %  of  the  time  do  you  use  the  evaluations 
during  the  pre-award  survey  process?)  were  matched  against 
the  responses  to  question  6  (In  the  past  year  how  many  con¬ 
tractors  did  your  office  withhold  awards  from  because  of  an 
unsatisfactory  record  of  past  performance?)  These  results 
show  that  offices  which  used  the  evaluations  more  often  were 
able  to  support  more  contract  award  denials: 

Question  6 — Number  of  contractors  denied  awards  last  year 

0  12  4  6 

Question  1 — What  %  of:  Less  than  10% —  83000 
the  time  do  you  use 

evaluations  during  More  than  10% —  42021 

pre-award  surveys? 

The  offices  who  use  the  performance  evaluations 
more  often  during  the  pre-award  survey  process  reported  a 
higher  incidence  of  denying  contractors  awards, 
c.  Communication 

Wells  says  employees  should  be  told  the  purpose 
and  uses  of  performance  appraisa  u  to  minimize  uncertainty 
and  resistance.  The  results  of  the  contractor  survey 
clearly  indicate  the  contractors  are  unaware  that  the  evalu¬ 
ation  system  even  exists  (22  of  32  had  never  received  an 
evaluation) .  Lawler  and  Rhode  say  those  being  evaluated 
should  receive  the  evaluations.  NAVFAC  currently  notifies 


contractors  of  their  performance  evaluation  only  if  the  per¬ 
formance  is  unsatisfactory  (6.4%  of  the  contracts). 

d.  Training 

Wells  says  that  appraisers  should  be  well- 
trained.  Only  38%  of  AROICC's  surveyed  said  they'd  been 
trained  in  completing  evaluations  (most  of  that  training  was 
informal) .  Without  common  training  NAVFAC  is  left  with 
AROICC's  who  hold  a  wide  variety  of  ideas  about: 

-  The  intent  of  each  rating  category 

-  The  amount  of  detail  needed  to  support  an  evaluation 

-  The  kinds  of  documentation  needed  to  support  an 
evaluation 

-  The  standards  of  performance  for  various  ratings 

This  wide  variety  of  ideas  reduces  the  relia¬ 
bility  of  the  evaluation  system.  Recall  that  reliability 
connotes  stability,  consistency,  objectivity,  dependability, 
and  absence  of  bias.  Decreased  reliability  leads  to 
decreased  credibility  when  the  SBA  reviews  the  performance 
evaluations . 

e.  Completeness 

Incomplete  measures  may  motivate  people  to  per¬ 
form  only  a  portion  of  the  behavior  needed  for  organization¬ 
al  effectiveness.  Standard  Form  1420  allows  contractors  to 
be  rated  on: 

-  Quality  of  Work 

-  Timely  Performance 

-  Effectiveness  of  Management 


-  Compliance  With  Labor  Standards 

-  Compliance  With  Safety  Standards 

While  these  broad  categories  cover  the  range  of 
important  elements  in  contractor  performance,  they  lack 
specificity,  and  therefore,  fail  to  provide  an  adeguate 
assessment  of  contractor  performance.  Contract  specialists 
(7  of  15),  AROICC's  (11  of  62),  and  supervisory  civil 
engineers  (2  of  7)  said  the  evaluations  would  be  more 
valuable  if  more  specific  evaluation  items  were  used. 
Though  these  numbers  don't  represent  a  majority  of  the 
survey  populations,  they  are  significant  because  the 
responses  were  made  to  an  open-ended  question.  Within  the 
context  of  the  model,  the  OICC  as  decision  maker  would 
receive  more  information  if  more  specific  items  were 
evaluated. 

f.  Requisite  Variety 

The  OICC  as  decision  maker  must  look  at  a  con¬ 
tractor's  record  of  past  performance  and  decide  whether  or 
not  to  give  him  another  award.  As  currently  used,  a 
marginal  contractor,  a  satisfactory  contractor,  and  a  very 
good  contractor  all  receive  a  satisfactory  rating.  The 
system  does  not  allow  a  distinction  between  those  levels  of 
performance.  Contract  specialists  (7  of  15)  and  AROICC's 
(11  of  62)  said  there  should  be  additional  ratings  (i.e., 
marginal  and  highly  satisfactory) .  These  were  also 


responses  to  an  open-ended  question.  These  extra  categories 


would  provide  OICC's  more  precise  information  about  a  con¬ 
tractor's  past  performance. 


g.  Valence 

To  what  degree  are  the  measurements  tied  to  the 
organizational  reward  system?  Of  20  offices  surveyed,  12 
withheld  no  contract  awards  due  to  unsatisfactory  contractor 
past  performance.  Only  3  of  20  withheld  more  than  1 
contract.  22  of  32  contractors  responding  had  never  seen  a 
performance  evaluation  before.  13  of  62  AROICC's  commented 
(in  response  to  an  open-ended  question)  that  the  evaluations 
were  not  useful  because  unsatisfactory  contractors  continue 
to  receive  awards.  3  of  62  said  NAVFAC  needs  a  meaningful 
reward  system  to  recognize  its  outstanding  contractors. 
Clearly  there  is  an  absence  of  valence  within  the  evaluation 
system. 


B.  USE  II — CONTROLLING  ONGOING  CONTRACTOR  PERFORMANCE 
1 .  Review 


NAVFAC' s  use  of  contractor  performance  evaluations 
to  control  ongoing  contractor  performance  is  analyzed  below. 
This  model  reproduced  from  Chapter  III  depicts  the  control 
system : 
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2 .  Analysis 

The  FAR,  DFARS ,  and  NAVFAC  P-68  do  not  acknowledge 
this  potential  use  for  contractor  performance  evaluations. 
2  of  the  5  EFD's  surveyed  (Pacific  and  Atlantic)  do  offer 
guidance  for  issuing  interim  evaluations  as  feedback  to  con¬ 
tractors.  When  analyzed  from  the  viewpoint  of  accepted  con¬ 
trol  theory,  NAVFAC 's  current  use  of  the  contractor 
performance  evaluation  system  to  control  ongoing  contractor 
performance  is  weak  in  these  areas, 
a.  Relevance 

Looking  backward,  relevance  implies  feedback 
value.  NAVFAC  notifies  a  contractor  of  his  performance 
evaluation  only  if  his  performance  was  unsatisfactory  (6.4% 
of  the  time).  That  notification  is  made  after  the  contract 
is  already  completed.  To  the  contractor  as  decision  maker 
(deciding  how  to  behave) ,  that  information  is  not  relevant 
(because  it  is  not  timely)  .  The  behavior  in  question  is 


completed  and  cannot  be  changed.  Within  the  context  of  this 
model  the  performance  evaluations  are  irrelevant. 


Interim  evaluations  would  provide  contractors 
relevant  information.  10  of  15  AROICC's  who've  used  interim 
evaluations  before  observed  a  noticeable  improvement  in 
contractor  performance.  13  of  19  contractors  said  they'd 
like  to  receive  interim  evaluations  because  they  provide 
feedback  on  their  performance. 

b.  Usefulness  For  Decision  Making 

18  of  19  contractors  said  they'd  like  to  receive 
copies  of  their  final  performance  evaluation.  The  contrac¬ 
tors  also  placed  a  fairly  high  value  on  their  evaluations 
they've  received  in  the  past  (average  7.9  on  a  scale  of  1  to 
10)  .  Although  the  final  evaluations  would  not  provide 
timely  information  about  their  performance  on  the  ongoing 
contract,  it  would  give  the  contractors  feedback  necessary 
to  adjust  their  performances  on  future  contracts. 

c.  Understandability 

Information  cannot  be  useful  to  decision  makers 
who  cannot  understand  it.  The  contractors  as  decision 
makers  must  be  able  to  understand  the  evaluations  they 
receive.  The  lack  of  standards,  training,  and  a  policy 
statement  to  contractors  about  the  uses  of  evaluations 
decreases  their  understandability. 


d.  Requisite  Variety 

This  factor  is  a  problem  within  the  context  of 
this  model  also.  "Individuals  tend  to  achieve  easy  goals 
but  not  exceed  them."  (Lawler  and  Rhode,  1976,  p.  42) 
Since  NAVFAC  rates  both  marginal  and  very  good  contractors 
as  satisfactory,  marginal  performance  has  become  the  "easy 
goal"  of  contractors.  With  more  rating  categories,  the 
marginal  performers  can  be  identified,  and  all  contractors 
can  strive  to  become  exceptional  or  outstanding. 

e.  Completeness 

Recall  that  incomplete  measures  motivate  people 
to  perform  only  a  portion  of  the  behavior  needed  for  organi¬ 
zational  effectiveness.  More  specific  rating  items  will 
provide  contractors  with  more  specific,  and  thus  more 
complete  feedback.  The  contractors  will  then  be  able  to 
focus  their  efforts  on  all  the  behaviors  needed  to  complete 
the  contracts  according  to  NAVFAC' s  standards. 

f.  Controllability 

The  contractor  has  control  of  his  own  behavior. 
However,  if  he  does  not  know  what  behavior  is  acceptable 
because  he  lacks  feedback,  the  evaluation  he  receives  is  not 
completely  within  his  control.  With  interim  evaluations  the 
contractor  would  have  some  very  definite  feedback  mid-way 
through  his  performance,  and  could  modify  his  behavior 
accordingly  to  improve  his  final  evaluation. 


c. 


USE  III — CONTROLLING  AROICC  PERFORMANCE 


1 .  Review 

Recall  in  this  model  that  the  AROICC  uses  informa¬ 
tion  from  contractors'  past  performance  evaluations  to  pre¬ 
dict  future  contractor  behavior.  That  future  behavior  can 
then  be  anticipated  and  better  managed  by  the  AROICC. 
Within  this  model  the  OICC  places  management  emphasis  on  the 
completion  and  use  of  contractor  performance  evaluations  by 
including  them  as  a  criteria  for  the  AROICC' s  personal  per¬ 
formance  rating.  This  model  reproduced  from  Chapter  III 
depicts  the  control  system. 


CONTROL  SYSTEM 


Performance 

Rating 


2 .  Analysis 

The  data  shows  that  only  38%  of  the  AROICC's  have 
referred  to  a  contractor's  past  performance  evaluations 
before  the  start  of  a  new  contract.  40%  of  AROICC's  said 
evaluations  are  stressed  by  management  only  when  they  are 
overdue  or  unsatisfactory.  45%  of  contract  specialists  said 
management  placed  less  than  a  fair  amount  of  stress  on 
evaluations.  When  analyzed  from  the  viewpoint  of  accepted 
control  theory,  NAVFAC ' s  current  use  of  the  contractor 
performance  evaluations  to  control  AROICC  performance  is 
weak  in  the  following  areas. 

a.  Usefulness  For  Decision  Making 


The  evaluations  can  be  used  for  many  purposes 
within  the  context  of  this  model.  Recall  that  60%  of  the 
AROICC's  surveyed  did  not  know  where  to  find  copies  of  the 
evaluations.  61%  of  the  AROICC's  never  refer  to  contrac¬ 
tors'  past  evaluations  prior  to  administering  a  contract. 
That  knowledge  about  a  contractor's  history  of  performance 
would  be  useful  when  developing  a  strategy  for  administer¬ 
ing  and  inspecting  a  contract.  Additional  attention  to  weak 
areas  of  performance  may  avoid  repeating  past  problems. 
These  comments  by  Schmode,  the  supervisory  civil  engineer  at 
New  London,  CT  provide  one  view  on  this  subject  (see 
Appendix  I  for  a  copy  of  Schmode 's  complete  memorandum): 


The  specific  report  required  by  reference  (a)  is  fine  for 
the  purpose  intended  but  does  not  provide  us  with  the 
information  that  would  prove  useful  in  fulfilling  our 
administrative  responsibilities  for  follow-on  contracts 
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with  the  same  contractor.  The  five  performance  elements 
identified  in  the  reference  (a)  required  report  (SF  1420) 
are  much  too  general  and  the  wide  dispersion  between 
"satisfactory"  and  "unsatisfactory"  leave  much  to  be 
desired.  Basically,  we  need  to  know  who  our  marginal 
contractors  are,  who  our  satisfactory  contractors  are  that 
have  some  marginal  or  unsatisfactory  traits.  If  we  know 
in  advance  of  the  specific  areas  in  which  our  contractors 
have  proven  themselves  to  be  less  than  satisfactory,  we 
can  give  our  attention  to  those  specific  areas  including 
emphasis  during  the  Preconstruction  Conferences.  In 
short,  we  need  a  system  that  can  be  accurate,  objective, 
consistent,  require  minimum  effort,  and  can  be  cumulative. 
(Schmode,  1985,  p.  1) 

The  remainder  of  Schmode ' s  memorandum  describes  such  a 
system. 

b.  Reliability 

26%  of  the  AROICC's  surveyed  did  not  know  their 
EFD's  had  an  instruction  covering  contractor  performance 
evaluations.  Only  38%  of  AROICC's  had  ever  received 
training  on  how  to  complete  an  evaluation.  These  facts 
indicate  there  may  be  a  lack  of  consistency,  stability,  and 
dependability  in  the  evaluations  prepared  by  those  AROICC's. 
If  OICC's  use  the  contractor  evaluations  as  a  rating  criter¬ 
ia  of  the  AROICC,  then  the  AROICC's  will  be  motivated  to 
prepare  higher  quality  evaluations.  These  higher  quality, 
more  reliable  evaluations  will  consequently  improve  the  use¬ 
fulness  of  the  evaluations  for  the  other  two  purposes  previ¬ 
ously  discussed. 

c.  Requisite  Variety  and  Completeness 

Increased  management  emphasis  on  evaluation 
preparation  will  cause  AROICC's  to  evaluate  contractors  more 
thoroughly  on  all  elements  of  their  performance.  The 


resultant  evaluations  will  improve  their  utility  for  their 
other  two  uses. 

d.  Valence 

When  asked  to  rate  the  evaluations ' s  effective¬ 
ness  for  improving  contractor  performance  a  majority  of  the 
AROICC's  said  the  evaluations  were  not  effective  (trimmed 
average  2.96  on  a  scale  of  1  (not  effective)  to  10  (very 
effective)).  40%  of  the  AROICC's  said  management  emphasizes 
the  evaluations  less  than  occasionally.  13  of  62  AROICC's 
said  the  evaluations  are  not  effective  because  contractors 
rated  unsatisfactory  continue  to  get  awards.  There  is  a 
general  impression  that  the  evaluations  do  not  make  any 
difference.  This  low  valence  level  adversely  affects  the 
quality  of  evaluations  prepared  and  the  use  of  evaluations 
by  the  AROICC's. 

D .  SUMMARY 

This  chapter  analyzed  each  of  the  three  possible  uses 
for  contractor  performance  evaluations  from  the  viewpoint  of 
accepted  control  and  measurement  theory.  The  next  chapter 
uses  that  analysis  as  a  basis  for  some  conclusions  and 
recommendations  about  NAVFAC's  current  use  of  construction 
contractor  performance  evaluations. 


VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  previous  five  chapters  included  the  background, 
theory,  data,  and  analysis  about  NAVFAC's  current  use  of 
contractor  performance  evaluations.  This  chapter  draws  some 
conclusions  about  that  use  and  makes  recommendations  on  how 
NAVFAC  can  improve  its  use  of  the  evaluations. 


A.  CONCLUSIONS 

After  reviewing  the  information  in  previous  chapters, 
the  following  conclusions  are  made  about  NAVFAC's  current 
use  of  contractor  performance  evaluations: 

1.  NAVFAC  lacks  standards  of  performance  which  clearly 
describe  the  difference  between  unsatisfactory,  satis¬ 
factory,  and  outstanding  performance. 

2.  The  constructions  contractors  are  not  generally  aware 
of  the  performance  evaluation  system. 

3.  The  evaluations  are  not  used  to  provide  contractors 
feedback  about  their  performances. 

4.  The  evaluations  are  not  timely,  because  they  are 
written  after  the  performance  is  completed. 

5.  AROICC's  are  not  well  trained  in  completing  the  evalu¬ 
ations.  The  lack  of  training  adversely  affects  the 
reliability  of  the  evaluations. 

6.  Management  emphasis  on  the  evaluation  process  could  be 
improved . 

7.  There  is  no  well  publicized  incentive  for  contractors 
to  provide  outstanding  performance. 

8.  The  evaluations  are  under-utilized  for  each  of  the 
three  possible  uses. 


9. 


The  Standard  Form  1420  is  not  specific  enough  in  iden¬ 
tifying  the  elements  of  contractor  performance  to  be 
evaluated . 


10.  The  data  base  of  information  from  the  evaluations  is 
inadequate . 


B.  SHORT-TERM  RECOMMENDATIONS 

These  recommendations  for  improving  NAVFAC's  contractor 

performance  evaluation  system  could  be  implemented  quickly. 

1.  Policy  Statement--NAVFAC  should  issue  a  policy  state¬ 
ment  describing: 

a.  The  uses  of  the  contractor  performance 
evaluations . 


b.  The  standards  of  performance  required  for  satis¬ 
factory,  outstanding,  and  unsatisfactory  perform¬ 
ance  ratings. 

c.  An  explanation  of  the  evaluation  elements. 

This  policy  statement  would  increase  contractor 
awareness  of  the  evaluation  system  and  improve  the  consis¬ 
tency,  completeness,  reliability,  and  usefulness  of  the 
evaluations.  A  draft  policy  statement  is  included  in 
Appendix  J. 

2 .  Interim  Evaluations — To  provide  contractors  timely 
feedback  about  their  contract  performance,  NAVFAC 
should  require  field  offices  to  issue  interim  evalua¬ 
tions.  The  Standard  Form  1420  can  be  used  with  a 
cover  letter  explaining  the  purpose  of  the  interim 
evaluations.  Some  field  offices  may  view  this 
requirement  as  excessive  due  to  their  already  heavy 
administrative  workload.  The  time  spent  preparing  the 
interim  evaluations  should  be  offset  by  time  saved 
when  improved  contractor  performance  decreases  the 
administrative  burden.  The  interim  evaluations  are 
also  useful  and  should  be  required  when  the  AROICC  on 
a  contract  is  replaced. 


3.  Contractor  Notif icat ion--NAVFAC  should  provide  all 
contractors  a  copy  of  their  performance  evaluations. 
The  feedback  value  of  these  evaluations  should  not  be 
wasted.  Knowing  the  evaluations  are  sent  to  the  con¬ 
tractors*  should  increase  the  AROICC' s  objectivity  when 


Training — Field  offices  should  train  AROICC's  on  how 
to  complete  evaluations.  The  AROICC's  should  also 
learn  the  purposes  and  standards  for  the  evaluations. 
This  training  can  be  conducted  at  the  field  level 
using  the  policy  statement  as  a  guide.  The  training 
will  improve  the  consistency,  completeness,  and  relia¬ 
bility  of  the  performance  evaluation  process. 

Management  Emphasis--Manaaement  personnel  should  rate 
their  AROICC's  on  how  well  they  prepare  and  use  the 
performance  evaluations.  This  increased  management 
emphasis  on  the  evaluations  will  improve  their  use  as 
a  source  of  information  about  prospective  contractor 
performance  histories.  AROICC's  will  take  more  care 
in  preparing  contractor  evaluations  if  they  know  their 
personal  performance  rating  is  at  stake. 

Reward  System — All  contractors  rated  outstanding 
should  receive  an  award  for  their  work.  The  awards 
should  be  meaningful  and  immediate.  The  contractors 
should  know  in  advance  what  award  is  given  for 
outstanding  per-f ormance .  In  the  short  term,  types  of 
awards  that  can  be  given  include: 

a.  Special  command  recognition  in  the  form  of  news¬ 
paper  articles,  special  ribbon  cuttings,  letter 
of  commendation,  etc. 

b.  Recommendation  to  other  clients  about  the  con¬ 
tractors  outstanding  work. 

Publicize  the  System — When  performance  evaluations  are 
used  to  deny  an  unsatisfactory  contractor  a  contract 
award  NAVFAC  should  publicize  it.  Too  many  people 
neglect  the  evaluation  system  because  they  feel  it 
doesn't  work. 

Justify  Satisfactory  Ratings — A  narrative  should  be 
required  for  satisfactory  evaluations  also.  AROICC's 
should  cite  a  contractor's  strong  and  weak  points  even 
if  the  rating  is  satisfactory.  That  information  is 
useful  for  predicting  performance  on  future  contracts. 
Not  requiring  a  narrative  on  satisfactory  ratings 
gives  AROICC's  pressed  for  time  a  quick  means  of  com¬ 
pleting  the  evaluation  requirement. 


LONG-TERM  RECOMMENDATIONS 

Modify  the  SF  1420--The  current  form  does  not  evaluate 
contractors  in  enough  specific  areas.  The  form  needs 
more  rating  categories  to  differentiate  the  marginal 
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from  the  satisfactory  and  highly  satisfactory  contrac¬ 
tors.  A  draft  modification  of  the  form  is  included  as 
Appendix  K. 


l  •  ' 


2.  Rewards--In  the  long  term  NAVFAC  should  be  able  to 

develop  some  tangible  and  meaningful  reward  for 
contractors  who  perform  outstanding  work.  The  con¬ 
tractor  community  could  be  consulted  for  suggestions. 


i>: 


3 .  Improve  the  Data  Base — Manual  record  keeping  of  con¬ 
tractor  performance  information  is  inadequate.  Very 
few  of  the  contract  specialists  surveyed  use  the  EFD 
files  of  contractor  performance  evaluations.  A  com¬ 
puterized  data  base  would  be  used  (6  of  7  supervisory 
civil  engineers  said  they  would  use  it)  .  The  data 
base  should  include: 

a.  A  summary  of  each  contractor's  evaluation  ratings 

b.  One  line  statements  of  contractor  strengths  and 
deficiencies . 

The  data  base  should  be  accessible  to  field  offices 
without  significant  assistance  from  EFD  personnel. 
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APPENDIX  A 


DEFINITIONS 


ARO ICC --Assist ant  Resident  Officer-in-Charge  of  Construc¬ 
tion.  Within  the  NAVFAC  system  this  term  refers  to  a  Civil 
Engineer  Corps  Officer  who  directly  administers  construction 
contracts.  Within  the  context  of  this  study  the  term  AROICC 
refers  to  both  military  and  civilian  personnel  who  directly 
administer  construction  contracts. 

Supervisory  Civil  Enqineer--The  senior  civilian  manager  of  a 
contracting  field  office.  A  senior  Civil  Engineer  Corps 
officer  is  normally  also  assigned  to  manage  in  most  NAVFAC 
field  offices. 

Contract  Specialist--Within  the  context  of  this  study  a 
contract  specialist  works  in  field  offices:  preparing  con¬ 
tracts  for  advertisement  and  award,  doing  pre-award  surveys, 
resolving  contractual  problems,  and  otherwise  supporting  the 
contracting  efforts. 

OICC — Officer-in-Charge  of  Construction.  Usually  a  senior 
Civil  Engineer  Corps  Officer  with  delegated  contracting 
authority.  By  virtue  of  that  authority  he  is  responsible 
for  the  success  of  all  contracting  actions  he  issues.  The 
OICC  usually  prepares  personnel  performance  evaluations  on 
AROICC' s  within  his  jurisdiction.  The  OICC  also  can  recom¬ 
mend  denying  a  contractor  an  award  based  on  his  determina¬ 
tion  that  the  contractor  is  not  responsible. 


APPENDIX  C 


AROICC  SURVEY 


1.  Approximately  how  many  satisfactory,  outstanding,  and 
unsatisfactory  contractor  performance  evaluations  have  you 
written  during  your  career  as  a  contract  administrator? 

Satisfactory  _  Outstanding  _  Unsatisfactory  _ 

2.  Does  your  EFD  have  an  instruction  governing  contractor 
performance  evaluations?  YES/NO 

If  yes,  how  much  help  does  it  provide  when  you  are 
filling  out  an  evaluation?  (CIRCLE  ONE) 

A  BIG  HELP  SOME  HELP  A  LITTLE  HELP  NO  HELP 

3.  Listed  below  are  some  possible  uses  of  performance 
evaluations.  Please  rank  them  in  order  from  (1)  meaning  the 
most  important  use,  to  (5)  meaning  the  least  important  use. 
(If  you  add  a  use  in  the  other  spot,  rank  from  1  to  6) 

_ Use  to  Provide  Feedback  to  the  Contractors 

_ Use  to  Support  Findings  of  Non-Responsibility,  and  thus 

Deny  Contractors  an  Award 

_ Used  by  Contract  Specialists  During  the  Pre-Award  Survey 

_ Use  to  Give  AROICC 's  Information  About  a  Prospective 

Contractor 

_ Use  to  Satisfy  the  FAR  Requirement  to  Complete  an 

Evaluation 

_ OTHER  (Please  Specify)  _ 

4.  Have  you  ever  received  any  training  on  how  to  complete 
the  performance  evaluation  form?  YES/NO 

If  yes,  which  of  these  statements  most  accurately 
describes  the  level  of  your  training?  (CHECK  ONE) 

_ Formal  Classroom  Training 

_ Briefed  during  ROICC  Staff  Meeting 

_ OJT  by  Contract  Specialist  or  another  AROICC 

I  _ Given  a  Copy  of  an  old  Evaluation  as  a  Guide 

_ Other  (Please  Specify)  _ 

5.  Do  you  ever  review  a  contractor's  record  of  past  per¬ 
formance  prior  to  the  start  of  a  new  contract?  (YES/NO) 
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If  yes,  approximately  how  often  do  you  do  so?  (CIRCLE 
ONE) 

Always  75%  of  50%  of  25%  of  5%  of 

the  time  the  time  the  time  the  time 

6.  Do  you  know  where  you  could  find  a  file  of  all  of  a 

particular  contractor's  past  performance  evaluations? 
YES/NO  Where?  _ 

7.  On  a  scale  of  1  to  10,  how  effective  do  you  think  the 

existing  performance  evaluation  system  is  at  improving  the 

overall  performance  by  NAVFAC  contractors?  (CIRCLE  A 

NUMBER) 

123456789  10 

Not  Effective  Very  Effective 

8.  Have  you  ever  written  an  interim  evaluation  on  a 

contractor  mid-way  through  a  project?  YES/NO 

If  yes,  what  effect  does  the  interim  evaluation  have  on 
the  contractor's  performance?  (CIRCLE  ONE) 

Improvement  Some  Improvement  Little  Improvement  No  Change 

9.  Do  you  think  interim  evaluations  would  help  you  to 
positively  influence  a  contractor's  behavior?  YES/NO 

Why?  _ 


10.  Which  of  these  statements  most  accurately  describes  how 
much  emphasis  your  office  management  places  on  the  perform¬ 
ance  evaluations?  (CHECK  ONE) 

_ Evaluations  Are  Stressed  Frequently 

_ Evaluations  Are  Stressed  Occasionally 

_ Evaluations  Are  Stressed  When  A  Contractor's  Performance 

is  UNSAT 

_ Evaluations  Are  Stressed  When  They're  Overdue  to  the  EFD 

_ Evaluations  Are  Never  Stressed 

11.  What  comments  do  you  have  about  the  way  we  evaluate  our 
contractors?  Please  include  any  suggestions  you  have  for 
improving  the  evaluation  process.  (Use  back  of  page  if  you 
need  more  room.) 


APPENDIX  D 


CONTRACTOR  'SURVEY 


1.  On  a  scale  of  1  to  10,  how  much  value  do  you  assign  to 
the  performance  evaluation  rating  you  receive  on  a  project? 
(CIRCLE  ONE) 

123456789  10 
No  Value  High  Value 

2.  Of  those  you've  received  from  NAVFAC,  how  objective  and 
well  documented  were  your  performance  evaluations?  (CHECK 
ONE) 


_ Very  Objective  and  Well  Documented 

_ Pretty  Objective  and  Well  Documented 

_ Not  very  Objective  or  Well  Documented 

_ Subjective.  No  Documentation 

3.  Of  the  performance  evaluations  you've  received  from 
NAVFAC,  did  they  provide  you  with  useful  feedback  about  your 
performance?  YES/NO 

If  YES,  how  was  the  feedback  useful?  _ 


4.  An  interim  evaluation  is  one  provided  mid-way  through  a 
project.  How  useful  do  you  think  interim  evaluations  vou I i 
be  in  providing  you  feedback  about  your  performance  or.  i 
contract?  (CHECK  ONE) 

_ VERY  USEFUL 

_ FAIRLY  USEFUL 

_ USEFUL 

_ OF  LITTLE  USE 

_ NOT  USEFUL 

5.  Would  you  like  to  see  NAVFAC  issue  interim  r-  in  • 

evaluations  on  its  contracts?  YE:'.  NO  WHY  ’ 


6.  Do  you  have  a  system  for  evaluating  your  subcontractors? 
YES/NO 


_ If  yes,  could  you  please  describe  your  system  or  enclose 

some  information  about  your  system  (i.e.,  rating  forms)? 


7.  Would  you  find  it  useful  to  receive  a  copy  of  your 
performance  evaluation  for  each  contract  you  perform? 
YES/ NO 

Why?  _ _ _ 


8.  What  comments  do  you  have  about  the  way  NAVFAC  evaluates 
its  contractors?  Please  include  any  suggestions  you  have 
for  improving  the  evaluation  process. 


APPENDIX  E 


CONTRACT  SPECIALIST  SURVEY 


1.  What  %  of  the  time  do  you  refer  to  a  contractor's  past 
performance  evaluations  to  assist  you  during  the  Pre-Award 
Survey  process? 

0%  10%  30%  50%  70%  90%  100% 

On  what  types  of  awards  (i.e.,  unknown  contractor,  MCON  job, 
etc.) 


2 .  What  %  of  the  time  do  you  discuss  the  pre-award  survey 
results  with  the  AROICC  assigned  to  the  project? 

0%  10%  25%  50%  70%  90%  100% 

Under  what  circumstances  do  you  do  so? 

3.  What  source/sources  do  you  use  to  find  copies  of  a 
contractor's  past  performance  evaluations?  (i.e.,  EFD  file, 
ROICC  file) 

4.  About  what  %  of  the  time  do  you  use  your  EFD's  past 
performance  evaluation  files  when  doing  a  Pre-Award  survey? 

0%  10%  25%  50%  75%  90%  100% 

Under  what  circumstances?  (i.e.,  MCON  job) 

5.  How  would  you  rate  the  quality  of  the  performance  evalu¬ 
ations  you've  seen  in  your  office? 

Do  you  think  they'd  stand  up  in  court  in  support  of  a  non¬ 
responsibility  determination? 

6.  How  much  emphasis  is  placed  on  the  evaluation  process 
within  your  office? 

_ A  LOT 

_ A  FAIR  AMOUNT 

_ SOME 

_ A  LITTLE 

_ NONE 
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7.  In  the  past  year,  how  many  contract  awards  have  you 
withheld  from  the  apparent  low  bidder  because  of  his  record 
of  past  performance? 


_ Was  it  easy  to  do?  Y/N  Why  not? 

Would  it  have  helped  if  you  had  3  or  4  well-documented 
unsats  to  reference  in  your  request  for  award  denial?  Y/N 

8.  What  comments  or  suggestions  for  improving  the  evalua¬ 
tion  process  do  you  have? 


Can  you  give  me  the  names  of 

1.  Sup  Civil  _  How  long  been  on  job 

2.  Civilian  Contract  Administrator  _ _ 

How  long  _ 
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APPENDIX  F 


APPENDIX  H 


SUPERVISORY  CIVIL  ENGINEER  SURVEY 


1.  How  well  does  your  EFD  instruction  governing  contractor 
performance  evaluations  help  the  AROICC's  when  they're 
filling  out  the  evaluations? 


2.  What  changes  would  you  suggest  to  improve  the  usefulness 
of  the  instruction? 


3.  Do  you  provide  your  AROICC's  any  training  on  how  to 
complete  the  performance  evaluation  form?  YES/NO  (Circle 
One) 


What  kind  of  training? 
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4.  Does  your  office  keep  separate  performance  evaluation 
files  on  each  contractor?  YES/NO  (Circle  One) 

On  a  scale  of  1  to  10,  how  effective  do  you  think  the 


vV 

existing 

performance 

evaluation 

system  is  at  improving  the 

•V 

A* 

overall 

performance 

by  NAVFAC 

contractors? 

(CIRCLE  A 

NUMBER) 

JWzMk 

1 

2  3  4 

5  6 

7  8  9 

10 

Not  Effective 

Very 

Effective 

f. 

6 .  Does 

your  office 

write  interim  evaluations 

on  contrac- 

tors  mid 

-way  through 

a  project? 

YES/NO 

If  yes,  what  do  the  interim  evaluations  usually  have  on 
the  contractor's  performance?  _ 


7.  Do  you  think  NAVFAC  should  issue  interim  evaluations  on 
all  of  its  construction  contracts?  YES/NO 


8.  What  do  you  think  of  the  form  we  use  for  contractor 
performance  evaluations  (SF  1420)?  Would  you  make  any 
changes  to  the  form,  and  if  so  what  changes? 


9.  If  you  had  access  using  the  office  computer  to  a  file 
listing  all  of  a  contractor's  past  performance  evaluations, 
would  you  use  it  to  get  a  feel  for  how  a  prospective  con¬ 
tractor  of  yours  usually  performs?  YES/NO 


10.  Do  you  think  we  should  give  our  contractors  a  copy  of 
their  performance  evaluations  at  the  end  of  a  contract? 
YES/NO  Why  or  why  not? 
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SCHMODE  MEMO  RAX  DC:'  ON  CQXTEACTCR 

performance  report  (cpr) 
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Memorandum 


OATt:  25  Nov  35 
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™:  Distribution  Cist 


*USJ  CONTRACTOR  PERFORMANCE  REPORT  (CPR) 

Ref:  (a)  NORTHNAVFACENGCOMINST  4335.5  Subj:  Performance  Evaluations  of 

Construction  Contractors 

Enel:  (1)  Sample  CPR  Porm 

1.  We  are  tasked  by  reference  (a!  to  evaluate  the  performance  of  our  various 

construction  contractors  at  the  conclusion  of  their  contract.  The  specific 
report  required  by  reference  (a)  is  fine  for  the  purpose  intended  but  does 
not  provide  use  with  the  the  information  that  would  prove  useful  in 

fulfilling  our  administrative  responsibilities  for  follow-on  contracts  with 
the  same  contractor.  The  five  performance  elements  identified  in  the 

reference  (a)  required  report  are  much  too  general  and  the  wide  dispersion 

between  ’satisfactory*  and  ’unsatisfactory*  leave  much  to  be  desired. 

Basically,  we  need  to  know  who  our  marginal  contractors  are,  who  our 
satisfactory  contractors  arc  that  have  some  marginal  traits  and  what  these 
marginal  or  unsatisfactory  traits  are.  If  we  know  in  advance  of  the  specific 
areas  in  which  our  contractors  have  proven  themselves  to  be  less  than 
satisfactory,  we  can  giva  our  attention  to  those  specific  areas  including 
emphasis  during  the  Preconstruction  Conferences.  In  short,  we  need  a  system 
that  can  be  accurate,  objective,  consistent,  require  a  minimum  of  effort,  and 
can  oe  cumulative. 

2.  Talkinq  to  a  numDer  of  you,  the  following  12  factors  seem  to  cover  the 
majority  of  the  significant  elements  that  should  be  our  concern: 

(1)  Superintendent. 

(2)  Safety. 

(3)  Timely  Completion. 

(4)  Shop  Qrawinqs. 

(5)  Completion  of  ’Punch  List*. 

(6)  As-Built  Drawings. 

(7)  Change  Order  Negotiations. 

(8!  Cooperation. 

(9)  DRIs. 

(10)  Payrolls. 

(11)  Quality  Assucance. 

(12)  Technical  Capability. 


A  brief  description  of  whav  is  intended  for  eacn  element  is  in  order. 


Superintendent.  On  site  when  work  in  progress;  coordinates  worn  of 
suocontractors;  expedites  work;  controls  project,  etc. 

Safety.  Site  cleanliness;  safety  meetings;  enforces  hacd-nats;  accident 
reports,  etc. 

timely  Completion.  Project  B.o.D.'d  within  current  contract  completion  date. 

Shop  Drawings.  timely  submission  and  complete;  contains  KR  certification 
stamp;  suomittal  log. 

Completion  of  Punch  list.  Completion  within  reason  after  30D ;  worxs  on  own 
initiative. 

As-Built  Drawings.  Keeping  current  as  job  progresses:  timely  submission  at 
BOD. 

Change  Order  Negotiations.  timely  response  to  RFPs;  reasonableness  of  cost 
breakdown;  willingness  to  negotiate  a  fair  and  reasonable  price  and  time. 

Cooperation.  Willingness  to  work  with  inspector  and  project  engineer. 

ORls.  timely  submission  and  completeness. 

Payrolls,  timely  submission  without  discrepancies. 

Quality  Assurance.  Workmanship  and  willing  compliance  with  plans  and  specs. 

Technical  Capability.  Competency  of  contractor  to  do  the  work;  ability  to 
provide  adequate  workmen  and  equipment. 

3.  A  simple  grading  system  that  is  self-explanatory  is  as  follows: 

Outstanding  -  5 
Above  Averaqe  -  4 
Satisfactory  -  3 
Marginal  -  2 

Unsatisfactory  -  1 

It  is  expected  that  a  majority  of  our  contractors  will  fall  in  the 
"Satisfactory*  category.  As  previously  stated,  it  is  those  contractors  that 
have  certain  elements  that  fall  in  the  'Unsatisfactory’  or  'Marginal* 
category  that  we  are  primarily  concerned  with.  At  the  time  we  complete  the 
more  formal  Construction  Contractor  Performance  Evaluation  Report,  the 
project  engineer  and  the  project  inspector  will  collectively  complete  this 
new  Contractor  Performance  Report  (CPR)  by  assigning  numerical  'grades*  to 
each  of  the  12  elements.  If  a  grade  is  either  1  or  2,  a  very  brief 
explanation  is  to  be  provided.  The  combined  effort  of  the  inspector  and  the 
project  engineer  should  not  require  much  over  15  to  20  minutes.  The  CPR  will 
be  prepared  for  all  construction  contracts.  The  cumulative  results  of  these 
reports  will  be  posted  in  our  computer  and  will  be  available  for  immediate 
recall  for  use  on  subsequent  contracts  with  the  contractor  involved. 
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4.  A  sample  of  the  CPR  form  to  oe  used  is  attached  as  enclosure  (1).  The  • 

form  provides  for  tine  indication  of  trie  general  classification  of  the  type  of 

project  involved,  e.g.,  electrical,  painting,  general,  etc.  Please  advise  if 

you  have  any  suggestions  on  how  to  improve  this  reporting  system  or  the  form 

iteself.  This  report  is  for  internal  use  only  amk  the  information  will 

neither  he  forwarded  to  Northern  Division  nor  /gi/ven  /Tlrectly  to  the 
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an  excessive  amount  of  government  administra¬ 
tive  effort  on  the  contract. 

Unsat isfactory — The  contractor  has  failed  to  meet  the  mini¬ 
mum  contract  requirements  even  after  signifi¬ 
cant  administrative  effort  by  the  government. 
The  contractor  was  uncooperative,  failed  to 
anticipate  problems  within  the  realm  of  his 
responsibility,  and  did  not  attempt  to  develop 
solutions.  The  contractors  actions  were  not 
professional . 

D .  EVALUATION  ELEMENTS — Within  the  5  broad  evaluation 
elements  on  the  SF  1420  contractors  will  be  specifically 
rated  on: 


Broad  Element _ Specific  Element 


A  . 

Quality  of  Work 

1. 

Materials  Used 

2  . 

Workmanship 

3  . 

Quality  Assurance  Plan  and 

Efforts 

B. 

Timely  Performance 

1. 

Scheduling 

2. 

Punch  List  Completion 

C. 

Effectiveness  of 

1. 

Effectiveness  of 

Management 

Superintendent 

2. 

Cooperation 

3  . 

Administration  (i.e.,  sub¬ 

mittals,  payrolls) 

D. 

Compliance  With 

1. 

Payrolls 

Labor  Standards 

2  . 

Violations 

r  _ 

Compliance  With 

1 . 

Safety  Awareness  and 

Safety  Standards 

Attitude 

2  . 

Safety  Plan 

3  . 

Safety  Record 

Although  these  specific 

items  may  have  been  implicitly 

i  lusted  using  the  SF  142 

0, 

they  will  now  be  explicitly 

•  •■/•I  1  it<-  i  using  this  modified  system.  The  contractors  and 


APOICC's  both  know  what  areas  of  performance  will  be 
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evaluated.  The  general  standards  can  be  applied  to  each 

specific  evaluation  element.  As  an  example,  the  following 

standards  could  apply  to  Quality  of  Work-Materials  Used: 

Outstanding  --All  materials  met  or  exceeded  specific  quality 
requirements.  Complete  and  acceptable 

material  submittals  were  made  as  required. 
Manufacturers  recommendations  and  appropriate 
industry  practices  were  followed  when  using 
all  materials.  Obvious  care  was  taken  when 
storing  and  handling  materials.  No  government 
effort  was  needed  to  ensure  compliance  with 
materials  requirements. 

Highly 

Satisfactory — All  materials  met  or  exceeded  specified  qual¬ 
ity  requirements.  Complete  and  acceptable 
material  submittals  were  made  as  required. 
Care  was  taken  when  storing  and  handling 
materials.  Manufacturers  recommendations  and 
appropriate  industry  practices  were  followed 
most  of  the  time.  Very  little  government 
effort  was  needed  to  ensure  compliance  with 
material  requirements. 

Satisfactory — All  materials  met  or  exceeded  specified  qual¬ 
ity  requirements.  Complete  and  acceptable 
submittals  were  made  as  required.  Some  resub¬ 
mittals  were  needed  before  final  material 
approval  was  granted.  Normal  care  was  usually 
taken  when  storing  and  handling  materials. 
Manufacturers  recommendations  and  industry 
practices  were  usually  followed.  Government 
effort  was  required  occasionally  to  ensure 
compliance  with  material  requirements. 

Marginally 

Satisfactory--All  materials  met  specified  quality  require¬ 
ments.  Complete  and  acceptable  submittals 
were  made  only  after  several  attempts.  Below 
average  care  was  taken  when  storing  and 
handling  materials.  Manufacturers  recommen¬ 
dations  and  appropriate  industry  practices 
were  occasionally  violated.  A  large  amount  of 
government  effort  and  some  rework  was  required 
to  ensure  compliance  with  material 
requirements . 

Unsat isfactory — Not  all  materials  met  specified  quality 
requirements.  Material  submittals  were 
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incomplete  and  often  unacceptable.  Little 
care  was  taken  when  storing  and  handling 
materials.  Manufacturers  recommendations  and 
appropriate  industry  practices  were  not 
routinely  followed.  A  significant  amount  of 
government  effort  and  rework  was  required  to 
bring  the  contractor  into  compliance  with 
material  requirements. 

E.  PREPARATION  AND  DISTRIBUTION:  When  reasonable  to  do  so, 
or  when  requested  by  a  contractor,  the  AROICC  will  issue  an 
interim  evaluation  at  the  mid-point  of  the  contract 
duration.  The  interim  evaluation  is  intended  to  provide 
contractors  feedback  about  their  performance.  The  contrac¬ 
tors  final  performance  evaluation  rating  is  intended  to  rate 
the  contractors  performance  over  the  entire  period  and 
therefore  should  not  depend  necessarily  on  the  interim 
evaluation  ratings. 

Interim  evaluations  should  be  sent  to  the  contractors  at 
the  mid-point  of  the  contract  duration.  If  a  contract  is 
for  an  extended  period  of  time,  interim  evaluations  should 
be  written  at  least  for  every  6  months  of  performance. 
Final  performance  evaluations  will  be  sent  to  contractors  at 
the  time  of  final  payment. 


APPEND  I  X  K 
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These  modifications  are  recommended  for  improvin'! 
Standard  Form  1420. 

1.  Items  i-S,  ii  and  1 2  shoal  1  remain  an  dim  ;*  i. 

the  vertical  space  allowed  tor  items  ■>,  ,  11,  an  i  1. 

be  reduced  to  accommodate  the  changes  to  items  a  and  1  . 

2.  Change  item  9  to  look  like  this: 


Perform-  Out- 


Elements 


H  igh 1 y 


stand-  Satisf, 


Satis-  Margi-  Unsatis¬ 
factory  nally  factory 

Satisf . 


A.  Quality  of  Work 

1 .  Mater l a  1 s 

2.  Workmanship 

3 .  Q/ A  plan 

B.  Timely  Performance 
1  .  Schedul ing 

2.  Punch  List 

C.  Effectiveness  of  Mgmt . 

1.  Superintendence 

2.  Cooperation 

1  .  Administratis n 

1  > .  C  o  m  p  1  l  a  n  c  e  w  I  ,;i  b  u  t 
1.  Payrolls 

2  .  V  i  o  1  at  1  on--. 

t  .  np  1  :  i  r,.  <>  w  a  ;  •  ■  *  v 

1  .  Aw  a  ren*"-.s  A*  *  ;  *  .  !«• 
■  1 1  »>’-  y  plan 
1  .  : .  I !  *'  t  ,•  r  •  r  i 
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